JP2006262465A - Apparatus and method for performing video synchronization by parsing a time stamp from a packet stored in a buffer - Google Patents

Abstract

An apparatus and method for performing video synchronization by parsing a time stamp from a packet stored in a buffer.
Storing a plurality of MPEG PES packets in a buffer; extracting a packet stored from the buffer; parsing the extracted packet to restore a timestamp; and a restored timestamp And a reference time, and a step of controlling decoding of the data of the packet extracted by the comparison result between the restored time stamp and the reference time. Therefore, it is not necessary to consider bidirectional coding, and only the PTS may be transmitted to the comparison / control unit, so that the operation can be simplified as compared with video transmission.
[Selection] Figure 3

Description

  The present invention relates to a digital compression method, and more particularly, to a method for synchronously decoding compressed digital information.

  The MPEG-2 standard, which is a standard for compressing moving images, is applied to a digital television (DTV) broadcasting system, and a DTV receiver to which this standard is applied has been developed. MPEG-2 is used in various other video and audio transmission devices.

  In general MPEG-2 transmission, an elementary stream (ES) (that is, an audio, video and / or data stream) is packetized, and a packetized elementary stream (PES). Form. The elementary stream (ES) is multiplexed together with other packetized elementary streams (PES) of the transport stream (TS). Here, the transmission stream (TS) refers to a stream that is continuously transmitted via a communication medium such as wireless, wired, or optical cable. In a video device, a packetized elementary stream includes a picture start code (PSC) and many types of time stamps. Here, the picture start code is a code for displaying the start of video data. Such a time stamp includes a DTS (Decoding Time Stamp) and a PTS (Presentation Time Stamp). DTS is a time stamp indicating a decoding time point, and PTS indicates a display time of encoded video. In general, DTS and PTS are all transmitted for use in I-pictures (Intra-Coded Pictures) and P-pictures (Predictive-coded Pictures). However, the PTS is transmitted for use in B-pictures (Bidirectionally predictive-coded pictures). A general decoder extracts elementary stream data and a time stamp from the PES. The extracted elementary stream data is stored in a buffer, and the extracted time stamp is stored in a memory. The time stamp is associated with a pointer to a buffer used to selectively output elementary stream data for decoding.

  A typical receiver structure for use in such a process is shown in FIG. The transport stream demultiplexer (Transport Stream Demultiplexer) 10 includes a transport stream depacketizer 11 and a PES demultiplexer 12. The transport stream depacketizer 11 multiplexes the input TS to create a PES. The PES demultiplexer 12 parses the PES packet header and stores the extracted header information (including the time stamp) in the time stamp memory 40. Further, the PES demultiplexer 12 stores the extracted elementary stream data in the ES buffer 20. The time stamp extraction unit 50 extracts a time stamp from the header information stored in the time stamp memory 40 and stores it. The extracted time stamp is transmitted to the comparison / control unit 60. The decoder 30 uses the time stamp for synchronization when decoding the elementary data stored in the ES buffer 20. The comparison / control unit 60 compares the reference time and an STC (System Time Clock) stored in the time stamp to decode the buffered elementary stream data and generate a control signal for display Used for. In response to such a control signal, the decoder 30 decodes the elementary stream stored in the ES buffer 20 and provides the decoded data to the display device 70. Accordingly, the display device 70 displays the decoded data on the screen.

In the system shown in FIG. 1, the time stamp is stored in the time stamp memory 40 so as to be associated with the storage position in the ES buffer 20 of the corresponding elementary stream data using the pointer as shown in FIG. Is done. The comparison / control unit 60 controls the elementary data to be output from the ES buffer 20 and transmitted to the decoder 30, so that the time stamp associated with the elementary stream data decoded from the time stamp memory 40 is used. And compare with STC. Similar contents are disclosed in Patent Document 1. Japanese Patent Application Laid-Open No. H10-228561 discloses a technique related to parsing an input packet and analyzing a time stamp.
US Pat. No. 6,091,769 US Pat. No. 5,667,980

  The technical problem of the present invention is to provide a method for processing a packetized code stream such as MPEG video or audio PES.

  According to the characteristics of the present invention for achieving the technical problem described above, a method for processing an encrypted stream includes a step of storing a plurality of packets of the encrypted stream in a buffer, a step of extracting the stored packet from the buffer, and an extraction. The packet is extracted by parsing the recovered packet and restoring the timestamp, comparing the restored timestamp with the reference time, and comparing the restored timestamp with the reference time. Controlling the decoding. Here, the step of pulling out the stored packet includes the step of pulling out the stored packet from the buffer according to the result of the preceding comparison in which the time stamp of the packet first extracted from the buffer is compared with the reference time. In one embodiment, the step of controlling the decoding of the packet data derived from the comparison result between the reference time and the recovered time stamp includes firstly selecting a packet corresponding to a time stamp indicating a time before the reference time. Decoding, decoding a packet corresponding to a time stamp indicating a time corresponding to the reference time, and decoding and buffering a packet corresponding to a time stamp indicating a time after the reference time. Including.

  In another embodiment, the cryptographic stream processing method includes storing a plurality of packets of the cryptographic stream in a buffer and sequentially processing the packets stored from the buffer. The sequential processing includes sequentially extracting packets stored from the buffer, parsing each extracted packet to restore a time stamp, and comparing each restored time stamp with a reference time. And a step of selectively decoding a packet extracted according to a comparison result between each restored time stamp and a reference time. Here, the step of sequentially processing the packets stored in the buffer includes the step of extracting the packet stored from the buffer according to the result of the preceding comparison in which the time stamp of the packet first extracted from the buffer is compared with the reference time.

  In another embodiment, the packetized cryptographic stream processing apparatus sequentially retrieves the stored packets from a buffer including a plurality of packets of the cryptographic stream, parses each extracted packet, and restores the time stamp. A processor is included that compares each restored timestamp with a reference time and selectively decodes a packet derived from the comparison result between each restored timestamp and the reference time. Here, the processor retrieves the packet stored from the buffer according to the result of the preceding comparison in which the time stamp of the packet first extracted from the buffer is compared with the reference time.

  In another embodiment, the storage medium storing the cryptographic stream processing program sequentially extracts the stored packets from a buffer including a plurality of packets of the cryptographic stream, and parses each extracted packet to restore the time stamp. Each of the restored time stamps is compared with a reference time, and a program code for selectively decoding a packet extracted based on a comparison result between each restored time stamp and the reference time is included. Here, the program code retrieves the packet stored from the buffer based on the result of the preceding comparison that compares the time stamp of the packet first extracted from the buffer with the reference time.

  The apparatus and method for performing the video synchronization by parsing the time stamp from the packet stored in the buffer of the present invention does not need to consider bidirectional coding, and may transmit only the PTS to the comparison / control unit. The operation can be simplified compared to video transmission.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be realized in various different forms. The present embodiment is intended to complete the disclosure of the present invention, and to those skilled in the art. The present invention is provided to fully inform the scope of the invention, and the present invention should be determined based on the description of the claims. Note that the same reference numerals denote the same components throughout the specification.

  FIG. 3 is a block diagram illustrating a receiver of a system (eg, a digital television system) for video encoding, transmission, and decoding according to an embodiment of the present invention. The encoded video elementary stream generated from the MPEG encoder 312 is packetized by the packetizing unit 314. The packetized elementary stream is multiplexed together with other streams by the transmission stream multiplexing unit 316 to become a transmission stream. Such work is well-known to those skilled in the art to which this technology belongs, and detailed description thereof is omitted.

  The transmission stream is transmitted to the transmission stream demultiplexing unit 322 via the transmission path 315. The transmission stream demultiplexing unit 322 is included in, for example, a receiver (receiver). The transmission path 315 may include, for example, a mixer, a transmitter, an antenna, a media converter, a down converter, and a decoder. In addition, the transmission path 315 includes a wireless, wired, or optical cable, a recording medium (for example, a magnetic storage medium, a semiconductor storage medium, an optical storage medium, etc.). The transport stream demultiplexer 322 recovers the PES by demultiplexing the TS. The PES packet is stored in the PES packet buffer 324. The PES packet stored in the PES packet buffer 324 is restored by the synchronization unit 326. The synchronization unit 326 restores the video data using the PES packet and transmits the video data to the decoding and video generation unit 328. The decoding and image generation unit 328 compares the time stamp included in the packet with the STC transmitted from the synchronization unit 326, and the decoding and reproduction of the input video data precedes in time, or Control so as not to slow down.

  FIG. 4 is a block diagram illustrating a video encoding, transmission, and decoding system including a receiver 400 according to another embodiment of the present invention. The receiver 400 includes a transmission stream demultiplexer 410. The transmission stream demultiplexing unit 410 demultiplexes TS to generate PES. The PES packet is stored in the PES packet buffer 420. The synchronization unit 430 receives the packet from the PES packet buffer 420 and extracts video data. In addition, the synchronization unit 430 transmits the extracted video data to the decoder 440. The synchronization unit 430 extracts a time stamp from the input packet and transmits it to the comparison / control unit 450. The comparison controller 450 generates a control signal for the decoder 440. The comparison / control unit 450 compares the time stamp with the system time clock and generates a control signal for the decoder 440. The decoder 440 decodes the payload in response to the control signal, and outputs the decoded video data to the video generation unit 460.

  FIG. 5 is a block diagram illustrating the receiver synchronization unit 430 ′ illustrated in FIG. 4 according to an exemplary embodiment of the present invention. The synchronization unit 430 ′ includes a syntax processor 432. The syntax processor 432 parses the PES packet output from the PES packet buffer 420 and extracts a header and a payload. The extracted header includes a time stamp and is stored in the header storage unit 434. The syntax processor 432 sends an interrupt to the microprocessor 436 when parsing the PES packet. The microprocessor 436 extracts a time stamp from the header stored in the header storage unit 434 and temporarily stores it in the time stamp storage unit 438. The stored time stamp is transmitted to the comparison / control unit 450.

  The time stamp (for example, PTS or DTS) transmitted to the comparison / control unit 450 varies depending on the type of video. For example, when bi-directional coding is not required (ie, when there is no B-picture), the microprocessor 436 extracts the PTS for the I-picture and the P-picture and transmits the PTS to the comparison / control unit 450. However, when bi-directional coding is required (ie when the PES packet contains a B-picture), the I-picture and P-picture must first be decoded. This is because I-pictures and P-pictures serve as source data for B-pictures. PTS indicates a point in time when video data is output by the decoder 440. DTS indicates a point in time when video data is decoded. Accordingly, the synchronization unit 430 extracts the DTS and / or PTS related to the I-picture and the P-picture and transmits the extracted DTS and / or PTS to the comparison / control unit 450. DTS and / or PTS is compared to STC.

  FIG. 6 is a flowchart illustrating a decoding operation according to another embodiment of the present invention. Referring to FIG. 6 together with FIGS. 4 and 5, a plurality of PES packets are stored in the PES packet buffer 420 (610). The PES packet is extracted from the PES packet buffer 420 (620). The syntax processor 432 parses the extracted packet and extracts header information and a payload. The syntax processor 432 stores the extracted header information in the header storage unit 434, and transmits the payload to the decoder 440 (630). The syntax processor 432 sends an interrupt to the microprocessor 436 during parsing (640). The microprocessor 436 extracts a corresponding time stamp from the header information stored in the header storage unit 438 and transmits it to the comparison / control unit 450 (650). The comparison / control unit 450 compares the time stamp with the STC (660), and controls the decoder 440 according to the comparison result. That is, the comparison / control unit 450 determines whether the decoder is to decode the video data or to skip or repeat (670). The syntax processor 432 sequentially extracts the packets stored in the buffer 420 and repeats the above-described processor (630 to 670).

  In one embodiment of the present invention, the syntax processor 432 extracts PES packets from the PES packet buffer 420 in a first-in first-out manner with a constantly increasing pointer. However, the method by which the syntax processor 432 extracts the PES packet from the PES buffer can be changed, and the PES packet can be selectively extracted based on the comparison result between the time stamp and the STC. For example, if it is determined that a packet at a time later than the comparison result STC is extracted from the PES packet buffer 420, the syntax processor 432 may skip the extraction of some packets in order to match the STC. In other words, the syntax processor 432 may increase the pointer value used when accessing the buffer to extract a rear-order packet that matches the STC. If it is determined that the packet at the time point earlier than the STC is extracted, the syntax processor 432 may hold the pointer value so that the same packet is output for a predetermined time.

  The comparison / control unit 450 controls the decoder 440 by another method. For example, when the comparison result time stamp is later than the STC (that is, when the time stamp is larger than the STC), the comparison / control unit 450 causes the decoder 440 to decode the video data (payload). Then, control is performed so that the output to the video generation unit 460 is delayed. Conversely, when the time stamp is earlier than the STC (that is, when the time stamp is smaller than the STC), the comparison / control unit 450 controls the decoder 440 to skip decoding of the video data. If the time stamp matches the STC, the comparison / control unit 450 causes the decoder 440 to decode the video data and output the decoded video data to the video generation unit 460. The advantage of such a scheme is that it can be individually compared with the STC while parsing the time stamp of the buffered PES packet. Therefore, it is possible to overcome the problem of synchronization error that can occur in the prior art in which a time stamp stored in a memory or the like is retrieved and extracted and compared with the STC.

  According to another embodiment of the present invention, the above-described processor can be similarly applied to an audio stream. FIG. 7 is a block diagram illustrating a system for performing audio encoding, transmission, and decoding according to another embodiment of the present invention. The audio elementary stream encoded by the MPEG encoder 712 is packetized by the packetizing unit 714 to become a PES. The transmission stream multiplexing unit 716 multiplexes the PES with other streams to generate a transmission stream.

  The TS is transmitted to the transmission stream demultiplexing unit 722 via the transmission path 715. The transmission stream demultiplexer is included in, for example, a receiver. The transmission path 715 can include, for example, a mixer, a transmitter, an antenna, a media converter, a down converter, a decoder, and the like. In addition, the transmission path 715 includes a wireless, wired, or optical cable, a recording medium (for example, a magnetic storage medium, a semiconductor storage medium, an optical storage medium, etc.). The transmission stream demultiplexing unit 722 restores the PES by demultiplexing the TS. The PES packet is stored in the PES packet buffer 724. The PES packet stored in the PES packet buffer 724 is restored by the synchronization unit 726. The synchronization unit 726 restores audio data using the PES packet and transmits the audio data to the decoding and sound generation unit 728. The decoding and sound generation unit 728 compares the time stamp included in the packet with the STC transmitted from the synchronization unit 726, and the decoding and reproduction of the input audio data precedes in time, or Control so as not to slow down.

  FIG. 8 is a block diagram illustrating an audio encoding, transmission, and decoding system including a receiver 800 according to another embodiment of the present invention. Receiver 800 includes a transmission stream demultiplexer 810. The transport stream demultiplexing unit 810 demultiplexes TS to generate PES. The PES packet is stored in the PES packet buffer 820. The synchronization unit 830 receives the packet from the PES packet buffer 820 and extracts audio data. In addition, the synchronization unit 830 transmits the extracted audio data to the decoder 840. The synchronization unit 430 extracts a time stamp from the input packet and transmits it to the comparison / control unit 850. The comparison control unit 850 generates a control signal for the decoder 840. The comparison / control unit 850 compares the time stamp with the system time clock and generates a control signal to the decoder 840. The decoder 840 decodes the payload in response to the control signal, and outputs the decoded acoustic data to the audio generation unit 860.

  FIG. 9 is a block diagram illustrating a receiver synchronization unit 830 ′ illustrated in FIG. 8 according to an embodiment of the present invention. The synchronization unit 830 ′ includes a syntax processor 832. The syntax processor 832 parses the PES packet output from the PES packet buffer 820 and extracts a header and a payload. The extracted header includes a time stamp and is stored in the header storage unit 834. During parsing of the PES packet, the syntax processor 832 sends an interrupt to the microprocessor 836. The microprocessor 836 extracts a time stamp from the header stored in the header storage unit 834 and temporarily stores it in the time stamp storage unit 838. The stored time stamp is transmitted to the comparison / control unit 850. Unlike the synchronization unit 430 ′ of FIG. 5, the audio synchronization unit 830 ′ does not need to consider bidirectional coding. The synchronization unit 830 ′ may transmit only the PTS to the comparison / control unit 850, and thus the operation is simplified as compared with video transmission. Hereinafter, the skip and repeat operations are similar to those described above.

  The decoding and decoding apparatus according to the present invention can be applied to decoding of MPEG program stream (PS) packets, and thus can also be applied to communication using packets according to the MPEG-2 standard.

1 is a block diagram illustrating an MPEG receiver according to the prior art. FIG. FIG. 2 is a schematic diagram illustrating time stamp storage and association of the receiver shown in FIG. 1 according to the prior art. 1 is a block diagram illustrating a digital television system according to an embodiment of the present invention. 1 is a block diagram illustrating a system for encoding, transmitting, and decoding video according to an embodiment of the present invention. FIG. 5 is a block diagram showing a receiver synchronization apparatus shown in FIG. 4 according to another embodiment of the present invention. 5 is a flowchart illustrating a packet processing operation of the receiver shown in FIG. 4 according to another embodiment of the present invention. 1 is a block diagram illustrating a system for encoding, transmitting, and decoding audio according to an exemplary embodiment of the present invention. FIG. 8 is a block diagram illustrating a receiver synchronization apparatus illustrated in FIG. 7 according to another embodiment of the present invention. FIG. 8 is a block diagram illustrating a decoding operation of the receiver shown in FIG. 7 according to an embodiment of the present invention.

Explanation of symbols

312: MPEG encoder 314: Packetization unit 315: Transmission path 316: Transmission stream multiplexing unit 322: Transmission stream demultiplexing unit 324: PES packet buffer 326: Synchronization unit 328: Decoding and video generation unit

Claims (26)

Storing a plurality of packets of the cryptographic stream in a buffer;
Retrieving stored packets from the buffer;
Parsing the extracted packet to restore a time stamp; and
Comparing the restored timestamp with a reference time;
And a step of controlling decoding of the extracted packet data according to the comparison result between the restored time stamp and a reference time. The step of extracting the stored packet includes a step of extracting the stored packet from the buffer according to a pre-comparison result obtained by comparing a time stamp of the packet first extracted from the buffer with the reference time. The encryption stream processing method according to claim 1. The step of extracting a packet stored from the buffer according to a result of preceding comparison comparing the time stamp of the packet first extracted from the buffer and the reference time includes the time stamp of the packet first extracted from the buffer and the reference time. The encryption stream processing method according to claim 2, further comprising: controlling a pointer used to access the buffer according to a preceding comparison result obtained by comparing Retrieving the stored packet from the buffer according to a result of preceding comparison comparing the time stamp of the packet first extracted from the buffer and the reference time includes re-extracting the stored packet. The cryptographic stream processing method according to claim 2. The step of controlling the decoding of the extracted packet data according to the comparison result between the reference time and the restored time stamp includes first decoding a packet corresponding to a time stamp indicating a time before the reference time. The cryptographic stream processing method according to claim 1, further comprising: The step of controlling the decoding of the extracted packet data according to the comparison result between the reference time and the restored time stamp includes decoding a packet corresponding to the time stamp indicating a time corresponding to the reference time. The cryptographic stream processing method according to claim 1, further comprising: coding. The step of controlling decoding of the extracted packet data according to a comparison result between the reference time and the restored time stamp includes decoding a packet corresponding to a time stamp indicating a time after the reference time. The encryption stream processing method according to claim 1, further comprising: buffering. The encryption stream processing method according to claim 1, wherein the encryption data stream includes a PES, and the reference time includes a system time clock. The method of claim 1, wherein the encrypted data stream includes video, audio and / or video or audio data stream. Storing a plurality of packets of the cryptographic stream in a buffer;
Sequentially processing packets stored from the buffer;
The step of sequentially processing includes:
Sequentially retrieving packets stored from the buffer;
Parsing each of the extracted packets to restore a time stamp; and
Comparing each restored timestamp with a reference time;
And a step of selectively decoding the extracted packet according to the comparison result between each of the restored time stamps and the reference time. The step of sequentially processing the packets stored in the buffer includes the step of extracting the packet stored from the buffer according to a result of preceding comparison that compares the time stamp of the packet first extracted from the buffer with the reference time. The packetized codestream processing method according to claim 10, wherein: The step of extracting a packet stored from the buffer according to a result of preceding comparison comparing the time stamp of the packet first extracted from the buffer and the reference time includes the time stamp of the packet first extracted from the buffer and the reference time. The packetized codestream processing method according to claim 11, further comprising: controlling a pointer used to access the buffer according to a preceding comparison result obtained by comparing The packetized code stream processing according to claim 10, wherein the selectively decoding step includes first decoding a packet corresponding to a time stamp indicating a time before the reference time. Method. The packetized codestream processing according to claim 10, wherein the selectively decoding step includes decoding a packet corresponding to a time stamp indicating a time corresponding to the reference time. Method. The packetized packet of claim 10, wherein the selectively decoding comprises decoding and buffering a packet corresponding to a time stamp indicating a time after the reference time. Code stream processing method. Retrieving the continuously stored packets includes re-retrieving the stored packets from the buffer according to a result of a pre-comparison that compares the time stamp of the first extracted packet from the buffer with the reference time. The packetized code stream processing method according to claim 10. The packetized code stream processing method according to claim 10, wherein the encrypted data stream includes a PES, and the reference time includes a system time clock. The packetized codestream processing method according to claim 10, wherein the encrypted data stream includes video, audio and / or video or audio data stream. Means for storing a plurality of packets of the cryptographic stream in a buffer;
Means for sequentially retrieving packets stored from the buffer;
Means for parsing each of the extracted packets to restore a time stamp;
Means for comparing each restored timestamp with a reference time;
A packetized cipher stream processing apparatus comprising: means for selectively decoding the extracted packet according to the comparison result between each restored time stamp and the reference time. The means for sequentially extracting the stored packets from the buffer includes means for extracting the stored packets from the buffer according to a result of preceding comparison that compares the time stamp of the packet first extracted from the buffer with the reference time. The packetized cryptographic stream processing apparatus according to claim 19, wherein The packetized encrypted stream processing apparatus according to claim 19, further comprising the buffer. Sequentially retrieving packets stored from a buffer including a plurality of packets of a cryptographic stream, parsing each of the extracted packets to restore a timestamp, comparing each of the restored timestamp with a reference time, and A packetized cipher stream processing apparatus comprising: a processor that selectively decodes the extracted packet according to a comparison result between each restored time stamp and the reference time. 23. The packetized packet of claim 22, wherein the processor retrieves a packet stored from the buffer according to a result of preceding comparison that compares a time stamp of a packet that is first extracted from the buffer and the reference time. Cryptographic stream processing device. The packetized cipher stream processing apparatus according to claim 22, further comprising the buffer. Sequentially retrieving packets stored from a buffer including a plurality of packets of a cryptographic stream, parsing each of the extracted packets to restore a timestamp, comparing each of the restored timestamp with a reference time, and A storage medium for storing a packetized encrypted stream processing program, comprising: a program code for selectively decoding the extracted packet according to a comparison result between each restored time stamp and the reference time. 26. The packetized packet of claim 25, wherein the program code retrieves a packet stored from the buffer according to a result of a prior comparison that compares a time stamp of the packet first extracted from the buffer with the reference time. A storage medium for storing the encrypted stream processing program.

Images