JP2008219589A - Method and apparatus for synchronously storing and reproducing media multiplexed data - Google Patents

Abstract

Media multiplexed data (audio data and video data) is received and stored so that it can be reproduced without deviation in the same format as when received.
Media multiplexed data received at a predetermined time interval by a transmission / reception unit is separated into audio data and video data by a separation unit. The time stamp assigning unit 103 assigns a time stamp to the separated audio data and video data at each reception timing, and the media control unit 104 combines the audio data and video data with the time stamp into one media synchronization file. Is stored in the recording media unit 105. At the time of reproduction, the multiplexing unit 108 connects and multiplexes the stored audio data and video data according to the assigned time stamp, and reproduces and transmits the same media multiplexed data as at the time of reception.
[Selection] Figure 1

Description

  The present invention relates to a technique for storing and reproducing media multiplexed data, and more particularly, to a method and apparatus for synchronously storing and reproducing multiplexed media data.

  In everyday conversation and business situations, it is often desirable to confirm the contents of conversations or meetings conducted using so-called videophone terminals at a later date. For this purpose, it is desirable to store video / audio data of a videophone and faithfully reproduce the video and audio at that time. In this case, it is an important issue how to store and reproduce the voice and video of a speaker speaking on a videophone with the same quality as when stored.

  In the audio and image transmission / reception recording system disclosed in Japanese Patent Laying-Open No. 2005-057362 (Patent Document 1), a multiplexed data stream of moving image / audio data is received from a videophone device at the other end, and is converted into moving image data. After separation into audio data, they are multiplexed again and recorded as one data in the storage unit. When the multiplexed data stream is separated into moving image data and audio data, it is desirable to add a time stamp to each frame in order to synchronize the moving image and audio. The time stamp is calculated so that the time required for the decoding process differs between the moving image and the sound, and the moving image and the sound are synchronized (see paragraphs 0042-0043). When reproducing the recorded multiplexed data, the multiplexed data is read with reference to the time stamp, separated into moving image data and audio data, and then decoded and reproduced (see paragraphs 0045-0046). .

  As described above, the system disclosed in Patent Document 1 employs a method in which the received multiplexed data is separated and then video data and audio data are multiplexed again and recorded, and the time at that time is used. The stamp is added to compensate for the difference in time required for decoding. Therefore, in order to reproduce, it is necessary to separate the read multiplexed data and decode and reproduce the separated moving image data and audio data as in the case of reception.

JP 2005-057362 A

  As disclosed in Patent Document 1, the method of multiplexing and recording the separated moving image data and audio data again is considered to facilitate synchronization of the moving image and audio during reproduction. This is because if audio and moving images are recorded separately, the sound and video are not synchronized during playback.

  However, in the above-described prior art, in order to record / reproduce multiplexed audio / video data, received data is separated and then multiplexed and recorded, and further separated at the time of reproduction. In addition to the demultiplexing / multiplexing unit, another demultiplexing / multiplexing unit needs to be provided, resulting in an increase in circuit scale and a decrease in processing speed. Such an increase in circuit scale should be avoided especially for portable terminals that have to be reduced in size and weight, and for portable terminals with limited processing capabilities, it should also be avoided to place an extra load on the CPU. It is.

  It is an object of the present invention to provide a method and apparatus that enables synchronized storage and reproduction of media multiplexed data without increasing the circuit scale and processing load.

  Another object of the present invention is to provide an apparatus that enables synchronized storage and reproduction of media multiplexed data without increasing the circuit scale and processing load of the mobile terminal.

  According to the present invention, received media multiplexed data is separated for each medium, a time stamp is given to each media data, and each media data to which a time stamp is given is recorded. At the time of reproduction, the media data recorded in accordance with the time stamp is multiplexed to reproduce the original media multiplexed data.

  According to one aspect of the present invention, an apparatus for synchronously accumulating and reproducing media multiplexed data obtained by multiplexing a plurality of media data separates the media data from media multiplexed data received at a predetermined time interval. Separating means for performing, a time stamp giving means for giving a time stamp corresponding to the reception timing to each data of the plurality of media, and a storage for storing the data of the plurality of media given the time stamp for each medium And multiplexing means for multiplexing the data of the plurality of media read from the storage means according to a time stamp given thereto, and generating media multiplexed data identical to the received media multiplexed data. It is characterized by that.

  It is desirable that the time stamp assigning means assigns a time stamp corresponding to a relative elapsed time with reference to a leading reception timing of a series of media multiplexed data. Furthermore, it is desirable that the time stamp assigning unit also attaches an association flag indicating the presence / absence of data of another medium corresponding to the data of one medium to each data of the plurality of media. In this case, the multiplexing means can efficiently multiplex the data of the plurality of media according to the time stamp given to them while checking the association flag.

  The synchronous storage method of the present invention separates the data of the plurality of media from the media multiplexed data received at the timing of a predetermined time interval, and sets a time stamp corresponding to the reception timing for each data of the plurality of media. The plurality of media data to which the time stamp is applied is stored in the storage means for each medium.

  The synchronized playback method of the present invention reads the data of the plurality of media read from the storage means, reads the time stamp assigned to them, and reads the data of the plurality of media read according to the time stamp read in the previous period Is multiplexed to reproduce the same media multiplexed data as the received media multiplexed data.

  According to one embodiment of the present invention, an audio / video multiplexed data synchronous storage / playback apparatus connected to a plurality of videophone terminals through a call center transmits / receives audio / video multiplexed data to / from any videophone terminal through the call center. Transmitting / receiving means for performing, separating means for separating audio data and video data from audio / video multiplexed data received at a predetermined time interval, respectively, and receiving timing for each of the audio data and video data Time stamp giving means for giving a corresponding time stamp, storage means for storing the audio data and video data to which the time stamp is given respectively, and the audio data and video data read from the storage means to them Multiplexed according to the given time stamp and received Audio data and video data of the audio / video multiplexed data received from the first videophone terminal in response to a storage request from the first videophone terminal in response to a storage request from the first videophone terminal Are stored in the storage means together with a time stamp, and the same media multiplexed data as the received media multiplexed data read out from the storage means in response to a reproduction request for the audio / video multiplexed data from the second videophone terminal Media control means for transmitting to the second videophone terminal.

  As described above, according to the present invention, received media multiplexed data is separated for each medium, and each media data is given a time stamp according to the reception timing. Record each. At the time of reproduction, by multiplexing the data of each medium according to the time stamp, it is possible to reproduce the same media multiplexed data as at the time of reception with no time deviation between the corresponding media. Furthermore, efficient reproduction can be performed by using the association flag in addition to the time stamp.

  In particular, by connecting the videophone terminal and the audio / video multiplexed data storage / playback apparatus according to the present invention through a network or call center, the audio / video multiplexed data can be faithfully reproduced without increasing the circuit scale and processing load of the videophone terminal. Can be stored and played back. In this case, there is an advantage that audio / video multiplexed data can be recorded and reproduced without depending on the type of the videophone terminal.

1. Functional Configuration FIG. 1 is a block diagram showing a functional configuration of a storage / reproduction device according to an embodiment of the present invention. Here, in order not to complicate the explanation, the storage / reproduction device 10 is connected to a plurality of videophone terminals 30 through a network or a call center 20, and transmits / receives media multiplexed data (here, audio / video multiplexed data). To do.

  The storage / reproduction device 10 can transmit / receive multiplexed audio / video data to / from any videophone terminal 30 through the network or the call center 20 by the transmission / reception unit 101. The audio / video multiplexed data received from the videophone terminal 30 that requested the recording is separated into audio data and video data by the separation unit 102, and further, the audio data and video data are recorded by the time stamp assigning unit 103 as described later. Each is assigned a time stamp and an association flag. The time stamp, the audio data with flag, and the video data are respectively stored in the recording media unit 105 by the media control unit 104. The recording medium unit 105 is a recording medium such as a hard disk.

  In addition, when there is a playback request from a certain videophone terminal 30, the media control unit 104 reads the corresponding audio data and video data from the recording media unit 105, and the time stamp and flag are read by the time stamp reading unit 107. It is done. Then, according to the read time stamp and flag, the multiplexing unit 108 multiplexes the audio data and the video data in synchronization, reproduces the same audio / video multiplexed data as received, and the transmission / reception unit 101 requests to reproduce it. It transmits to the videophone terminal 30 where there was.

  The separation unit 102 and the multiplexing unit 108, the time stamp assigning unit 103, the time stamp reading unit 107, and the media control unit 104 may be configured as hardware, but each program is executed on a program control processor such as a CPU. The same function can be realized by executing the above. Hereinafter, the operation of the present embodiment based on the software configuration will be described in detail with reference to FIGS.

2. Operation FIG. 2 is a software configuration diagram showing a data flow at the time of storage, FIG. 3 is a software configuration diagram showing a data flow at the time of reproduction, FIG. 4 is an explanatory diagram showing an operation sequence in the entire network system, and FIG. It is explanatory drawing which shows the data structure in each time at the time of accumulation | storage reproduction | regeneration. The outline of the data flow during storage and playback will be described with reference to FIGS. 2 and 3. In FIGS. 2 and 3, the separation unit 102, time stamp assigning unit 103, time stamp reading in FIG. The unit 107 and the multiplexing unit 108 are collectively shown as “multiplexing / separating unit”.

  In the accumulation direction shown in FIG. 2, the media multiplexed data from the videophone terminal 30 is separated by the multiplexing / separating unit, and a time stamp is given to the separated audio data and video data at each reception interval, and the media control unit The data is stored in the recording medium unit 105 through 104. In the playback direction shown in FIG. 3, the audio data and the video data read from the recording media unit 105 by the media control unit 104 are combined and multiplexed according to the time stamp by the multiplexing / demultiplexing unit and transmitted to the videophone terminal 30. . In this way, by providing time stamps corresponding to video and audio at the time of separation, audio data and video data can be synchronously stored and reproduced.

2.1) Operation during storage (right arrow in Fig. 4)
Storage processing is started when media multiplexed data transmitted from a certain videophone terminal 30 is received by the transmission / reception unit 101. In this embodiment, the media multiplexed data is configured by multiplexing audio data and video data following the synchronization flag and header as shown in FIG. 5A. (Example: text data) may be multiplexed, and in that case, processing can be performed in substantially the same manner as described below.

  The multiplexed media data received by the transmission / reception unit 101 is input to the separation unit 102. The separation unit 102 separates the media multiplexed data input from the transmission / reception unit 101 into audio data and video data, and passes them to the time stamp assignment unit 103 respectively. For each of the audio data and the video data sequentially passed from the separation unit 102, the time stamp assigning unit 103 has an A / V flag indicating an association between the time stamp and the audio / video for each reception interval (for example, 20 ms). Is granted.

  For example, as shown in FIG. 5B, if the reception interval is 20 ms, the time stamp given to a certain audio data 1 and the corresponding video data 2 is TS00, the subsequent audio data 3 and the corresponding video data. The time stamp given to the data 4 is given a time stamp such as TS20.

  Further, the time stamp assigning unit 103 displays a V flag indicating whether or not corresponding video data exists for the audio data, and A indicates whether or not corresponding audio data exists for the video data. Each flag is given together with a time stamp. That is, as shown in FIG. 5B, “V flag = 0” for audio data without corresponding video data, “V flag = 1” for audio data with corresponding video data, and corresponding audio data. “A flag = 0” is assigned to video data having no data, and “A flag = 1” is assigned to video data having corresponding audio data.

  FIG. 5B shows the structure of the audio data and the video data to which the time stamp and the A / V flag are assigned in this way.

  The audio data and video data to which the time stamp and the A / V flag are assigned are passed to the media control unit 104. The media control unit 104 creates audio data and video data with a time stamp as a single media synchronization file, and the recording media unit 105 stores the media synchronization file in a predetermined storage area. The file format at this time is shown in FIG.

2.2) Operation during playback (left arrow in FIG. 4)
When a request for audio / video reproduction from a certain videophone terminal 30 is received by the transmission / reception unit 101, reproduction processing is started.

  First, the media control unit 104 reads the corresponding media synchronization file from the recording media unit 105 and passes the audio data and video data to the multiplexing unit 108. These audio data and video data are given a time stamp and an A / V flag as shown in FIG. These time stamps and flags are read by the time stamp reading unit 107.

  The multiplexing unit 108 generates the same media multiplexed data as that at the time of reception by connecting the time stamps of the audio data and the video data so as to be synchronized according to the time stamp and the flag read by the time stamp reading unit 23, and the transmission / reception unit 101. The multiplexing unit 108 can efficiently know the presence or absence of the corresponding audio or video data by checking the A flag and V flag of each video and audio data. The structure of the audio and video data multiplexed in this way is shown in FIG.

  The transmission / reception unit 101 adds predetermined header information or the like to the media multiplexed data passed from the multiplexing unit 108, and then transmits the media multiplexed data to the videophone terminal 30 that issued the reproduction request. FIG. 5E shows the configuration of the transmitted media multiplexed data.

  As described above, in the storage / playback apparatus according to the present embodiment, the same media multiplexed data received and stored from an arbitrary videophone terminal 30 is transmitted in response to a request from the same or another videophone terminal 30. can do. In addition, it is possible to achieve light synchronization storage and reproduction without providing a separation / multiplexing unit separate from the separation unit 102 and the multiplexing unit 108 and without performing heavy processing.

3. Application Example FIG. 6 is a block diagram showing an example of a system using the storage / playback apparatus of this embodiment. The basic configuration of the synchronous storage / synchronous playback apparatus 10 is the same as that in FIG. 1, but the video data played back by the videophone terminal 30 that has issued the playback request is different. In this example, another audio / video data prepared in advance is added and reproduced. For example, it is possible to provide a service in which a commercial “This service will be sent with xx provision” is added to the audio / video to be played.

  FIG. 7 is a block diagram showing another example of a system using the storage / playback apparatus of this embodiment. The basic configuration of the synchronous storage / synchronous playback apparatus 10 is the same as that shown in FIG. 1, but the transmission destination of the audio / video data is different. As shown in the figure, the transmission destination of the audio / video data can be the personal computer 201 or the Internet 202. In this way, transmission to not only the videophone terminal 30 but also to an unspecified number is possible, and information can be shared and exchanged with more people.

  The present invention is applicable to a multimedia communication system that requires synchronization, such as a videophone communication system or a video conference system.

The block diagram which shows the functional structure of the storage / reproducing apparatus by one Embodiment of this invention. The software block diagram which shows the flow of the data at the time of accumulation | storage. The software block diagram which shows the flow of the data at the time of reproduction | regeneration. Explanatory drawing which shows the operation | movement sequence in the whole network system. Explanatory drawing which shows the data structure in each time at the time of data storage reproduction | regeneration. The block block diagram which shows an example of the system using the storage / reproducing apparatus of this embodiment. The block block diagram which shows the other example of the system using the storage / reproducing apparatus of this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Separation part 13 Time stamp provision part 15 Media control part 17 Recording media part 21 Multiplexing part 23 Time stamp reading part 30 Transmission / reception part 50 Videophone terminal device 60 Call center

Claims (9)

An apparatus for storing and reproducing media multiplexed data obtained by multiplexing data of a plurality of media,
Separating means for separating the data of the plurality of media from media multiplexed data received at a predetermined time interval;
Time stamp giving means for giving a time stamp according to the reception timing to each data of the plurality of media;
Storage means for storing data of a plurality of media given the time stamp for each medium;
Multiplexing means for multiplexing the data of the plurality of media read from the storage means according to a time stamp given thereto, and generating media multiplexed data identical to the received media multiplexed data;
An apparatus for synchronously storing and reproducing media multiplexed data, comprising:   2. The synchronized accumulation of media multiplexed data according to claim 1, wherein the time stamp assigning unit assigns a time stamp corresponding to a relative elapsed time with reference to a leading reception timing of a series of media multiplexed data. Playback device. The time stamp assigning unit also assigns an association flag indicating the presence / absence of data of another medium corresponding to data of one medium to each data of the plurality of media,
The synchronized storage / reproduction of media multiplexed data according to claim 1 or 2, wherein the multiplexing means multiplexes the data of the plurality of media according to a time stamp given to them while checking the association flag. apparatus. A method of storing media multiplexed data obtained by multiplexing data of a plurality of media,
Separating the media data from the media multiplexed data received at a predetermined time interval;
A time stamp corresponding to the reception timing is given to each data of the plurality of media,
Storing data of a plurality of media to which the time stamp is given in a storage unit for each medium;
A method of synchronously storing media multiplexed data. A method of reproducing data of a plurality of media stored in the storage means by the storage method according to claim 4,
Read the time stamp given to the data of the plurality of media read from the storage means,
Reproducing the same media multiplexed data as the received media multiplexed data by multiplexing the read data of the plurality of media according to the time stamp read in the previous period.
A method for synchronously reproducing media multiplexed data. In an audio / video multiplexed data storage / playback device connected to multiple videophone terminals via a network,
Transmission / reception means for transmitting / receiving audio / video multiplexed data to / from any videophone terminal through the network;
Separation means for separating audio data and video data from audio / video multiplexed data received at a predetermined time interval;
A time stamp giving means for giving a time stamp corresponding to the reception timing to each of the audio data and the video data;
Storage means for storing each of the audio data and the video data to which the time stamp is given;
Multiplexing means for multiplexing the audio data and video data read from the storage means according to a time stamp given to them, and generating the same media multiplexed data as the received media multiplexed data;
In response to a storage request from the first videophone terminal, audio data and video data of the audio / video multiplexed data received from the first videophone terminal are stored in the storage means together with a time stamp, and the second videophone terminal Media control means for transmitting the same media multiplexed data as the received media multiplexed data read from the storage means to the second videophone terminal through the network in response to a reproduction request for the audio / video multiplexed data from ,
An apparatus for synchronously storing and reproducing audio / video multiplexed data.   7. A synchronous storage / reproduction system, wherein the audio / video multiplexed data synchronous storage / reproduction apparatus according to claim 6 is connected to the plurality of videophone terminals via a call center. Computer
Means for respectively separating the data of the plurality of media from the media multiplexed data received at a predetermined time interval;
Means for giving a time stamp corresponding to the reception timing to each data of the plurality of media;
Means for storing data of a plurality of media given the time stamp in a storage means for each medium;
A program for functioning as a synchronous storage device. An apparatus for reproducing data of a plurality of media stored in the storage means by the synchronous storage apparatus according to claim 4,
Means for reading time stamps attached to the data of the plurality of media read from the storage means;
Means for reproducing the same media multiplexed data as the received media multiplexed data by multiplexing the read data of the plurality of media according to the time stamp read in the previous period;
A program for causing a computer to function as a synchronized playback device.

Images