JPH0282887A - Picture voice transmission system - Google Patents

Abstract

PURPOSE:To automate the adjustment for ensuring a ripple sink by allowing a control section of both station equipments A, B to obtain an additional delay time and to set the result to a variable delay circuit based on a delay time of a picture data and a data of a delay time of a voice data stored in advance respectively. CONSTITUTION:An additional delay time arithmetic means 3 stores a delay time beta1 in a voice data sender side in-station processing and a delay time beta2 in a voice data receiver side in-station transmission processing section 10 as a fixed data in advance. An additional delay time arithmetic means 3 uses detection means 2, 9 so as to detect a delay time alpha1 in a picture data sender side in-station processing transmission section 1 and a delay time alpha2 in a picture data receiver side in-station processing transmission section 7 respectively. Then an additional delay time to be given additionally to a voice data is obtained as (alpha1+alpha2)-(beta1+beta2) and the result is set to a variable delay means 6. Thus, the adjustment for ensuring a ripple sink is implemented automatically in response to the processing capability of the receiver side equipment.

Description

[Detailed Description of the Invention] [Summary] One terminal station applies predetermined processing to each of image data and audio data corresponding to the image data, and transmits the processed data to the other terminal station. , regarding an image/audio transmission system that further performs predetermined processing on each of the transmitted image data and the audio data corresponding to the image data at the other terminal station; The purpose of this is to make it possible to automatically make adjustments to ensure lip synchronization according to the processing mode applied to image data on the receiving device and the processing capacity of the receiving device. an image data transmitting side internal processing/transmission unit that applies predetermined processing to the image data and transmits it to the receiving side station; and a predetermined process that applies predetermined processing to the audio data to be transmitted to the receiving side station corresponding to the image data to be transmitted. A transmitting station device further comprising an internal processing/transmission unit for transmitting audio data to the receiving station, and an image that performs predetermined processing on the image data transmitted from the transmitting station device and outputs the resultant image. Data receiving side intra-office transmission/
It consists of a processing unit and a receiving side station device having an audio data receiving side intra-office transmission/processing unit that applies predetermined processing to the audio data transmitted in correspondence with the transmitted image data and outputs the result. In the image/audio transmission system, the receiving side station device includes a receiving side intra-station delay time detection means for detecting a delay time in the image data receiving side intra-office transmission/processing section, and a receiving side intra-office delay time detection means for detecting a delay time in the image data receiving side intra-office transmission/processing section. receiving side station delay information transmitting means for transmitting a delay time to the transmitting side station device; a receiving side station delay information collection means for collecting the delay time of the audio data transmitting side; ,and,
The delay time in the internal transmission/processing section on the audio data receiving side is stored in advance as fixed data, and the delay time in the internal processing/transmission section on the audio data transmitting side and the internal transmission/processing section on the audio data receiving side and the image data transmission are stored in advance as fixed data. Delay time in the processing/transmission section within the side station and transmission/transmission within the receiving side station of the image data collected by the receiving side station delay information collection means.
additional delay time calculation means for calculating an additional delay time to be additionally given to the audio data to be transmitted to the receiving side station device from the delay time in the processing unit,
Furthermore, the apparatus is configured to include variable delay means for additionally providing the additional delay time to the audio data to be transmitted to the receiving side station device.

[Industrial application field]

According to the present invention, in one terminal station, image data and
Each of the audio data corresponding to the image data is subjected to predetermined processing and transmitted to the other terminal station, and the other terminal station also processes the transmitted image data and the image data. Furthermore, for each voice data to be supported,
The present invention relates to an image/audio transmission system that applies predetermined processing to each.

Generally, when transmitting image data, audio data corresponding to the image data is often also transmitted at the same time.

Furthermore, at this time, in order to reduce the number of data bits on the line, the transmitting side performs encoding processing on each of the image data and the corresponding audio data, and the receiving side performs encoding processing on each of the image data and the corresponding audio data. A process of decoding the encoded data is performed.

In particular, when transmitting image data, the amount of data to be transmitted is large, so depending on the characteristics of the image data, for example, full sampling and sub-sampling (sub-sas+pltng) are used. A plurality of encoding processing modes are provided, and one of the modes is selected.

By the way, when the encoding processing mode as described above changes, the time required for encoding and decoding processing of image data, that is, the delay time changes greatly.

Furthermore, due to differences in the decoding means of the receiving device,
The time required to process image data is different.

On the other hand, in the encoding and decoding processing of audio data, the time required for processing, that is, the delay time, does not usually change significantly as in the case of the above-mentioned image data. Therefore, for example, CCITT Y , 22
1 standard etc., when transmitting image data and audio data, a fixed length time slot is allocated to the audio data, while a variable length time slot is allocated to the image data depending on the encoding processing mode. bit length is allocated.

Therefore, the total delay time during processing such as encoding and decoding described above and during transmission for each of the image data and the audio data transmitted corresponding to the image data is made to match each other. [lip sync
5 ynch, )], a time lag will occur between the image and audio on the receiving side.

[Prior art and problems to be solved by the invention]

FIG. 6 is a schematic diagram of the conventional image/audio transmission system.

In FIG. 6, 101 is a transmitting station device, 201 is a receiving station device, 661 and 66! are transmission paths for transmitting image data and audio data from the transmitting station device 101 to the receiving station device 201, respectively.

Furthermore, in the transmitting side station device 101, 61 is an image data processing section, 62 is an image data transmitting side intra-office transmission line,
63 is an audio data processing section, 65 is an intra-office transmission line on the audio data transmitting side, and 64 is a variable delay means.

Further, in the receiving side station device 201, 67 is an image data receiving side intra-office transmission path, 68 is an image data processing unit, and 6
Reference numeral 9 represents an intra-office transmission line on the audio data receiving side, and 70 represents an audio data processing section.

In the transmitting station device 101, the image data processing unit 61 and the audio data processing unit 63 each perform predetermined processing such as encoding on input image data or audio data, as described above. The image data transmitting side intra-office transmission line 62 and the audio data transmitting side intra-office transmission line 65 each cover all the transmission paths through which image data or audio data is transmitted within the transmitting side station device 101. This is a representative example.

Similarly, in the receiving side station device 201, the image data processing section 68 and the audio data processing section 70 respectively perform processing on the received image data or audio data.
For example, the image data receiving side intra-office transmission line 67 and the audio data receiving side intra-office transmission line 69 perform predetermined processing such as the above-mentioned decoding. This is a representative representation of all transmission paths for transmission within the device 201.

Here, as described above, after image data is input in the transmitting side station device 101, the image data processing unit 61
, the time required for the image data to be outputted as an image signal via the transmission line 62, the transmission line 66, the intra-station transmission line 67, and the image data processing unit 68, and the receiving station apparatus. In 201, after audio data is input, it is transmitted through the audio data processing section 63, the audio data transmitting side intra-office transmission line 65, the transmission line 661, the audio data receiving side intra-office transmission line 69, and the audio data processing section 70. The time required to be output as an audio signal is -
must be defeated.

As shown in FIG. 6, conventionally, the transmitting side station device 101
In the audio data path, a variable delay means 64 is provided to give the audio data signal a delay time corresponding to a value set by manual input from the outside, and the delay time in the entire path of the image data is Adjustments were made by manual input so that the time and the delay time of the entire audio data path matched.

Therefore, in the past, when simultaneously transmitting image data and audio data, it was necessary to adjust the delay time to ensure lip sync, and when the partner station equipment changed, it was necessary to adjust the delay time to match the decoding processing time of the other party equipment. It is necessary to perform the delay time adjustment again. That is, there is a problem in that adjustments must be made each time the connected station changes.

The present invention has been made in view of the above-mentioned problems, and automatically ensures lip synchronization according to the processing mode applied to image data in the sending device and the receiving device, and the processing capacity of the receiving device. The object of the present invention is to provide an image/audio transmission system that makes it possible to make adjustments for.

[Means to solve the problem]

Figure 1 is a basic configuration diagram of the present invention.
00 is the transmitting side station device, 200 is the receiving side station device, and
111.11g and llz are transmission paths between the transmitting station device 100 and the receiving station device 200, respectively.

Furthermore, in the transmitting side station device 100, l is an image data transmitting side internal processing/transmission unit, 2 is a transmitting side internal station delay time detection means, 3 is an additional delay time calculation means, and 4 is a transmitting station delay information collection means. , 5 is an in-office processing/transmission section on the audio data transmitting side, and 6 is a variable delay means.

In the receiving side station device 200, 7 is an image data receiving side intra-station transmission/processing unit, 8 is a transmitting station delay information transmitting means, 9 is a receiving side intra-station delay time detecting means, and
IO is an in-office transmission/processing unit on the audio data receiving side.

The image data transmitting side internal processing/transmission unit 1 applies predetermined processing to the image data to be transmitted, and transmits the image data to the receiving station 200. The audio data transmitting side in-office processing/transmission unit 5 applies predetermined processing to the audio data to be transmitted to the receiving station 200 in correspondence with the image data to be transmitted, and transmits the data to the receiving station 200.

The image data receiving side intra-office transmission/processing section 7 applies predetermined processing to the image data transmitted from the sending side station device 100 and outputs the resultant image data.

The audio data receiving side intra-office transmission/processing unit 10 applies predetermined processing to the audio data transmitted in correspondence with the transmitted image data and outputs the resultant data.

The receiving-side intra-office delay time detection means 9 detects the delay time in the image data receiving-side intra-office transmission/processing section 7 .

The receiving side station delay information transmitting means 8 transmits the delay time in the image data receiving side intra-station transmission/processing section 7 to the transmitting side station device 1.
Send to 00.

The receiving side delay information collecting means 4 collects the delay time in the image data receiving side intra-office transmission/processing section 7 from the receiving side station device 200.

The transmitting-side intra-office delay time detection means 2 detects the delay time in the image data transmitting-side intra-office processing/transmission section 1 .

The additional delay time calculating means 3 stores in advance the delay time in the audio data transmitting side internal processing/transmission unit 5 and the delay time in the audio data receiving side internal transmission/processing unit 10 as fixed data, and calculates the delay time in the audio data transmitting side internal processing/transmission unit 10 as fixed data. Delay time in the intra-office processing/transmission unit l, delay time in the intra-office transmission/processing unit 7 on the image data receiving side collected by the receiving side delay information collecting means 4, and intra-office processing/transmission unit on the audio data transmitting side. 5 and the delay time in the audio data receiving side intra-office transmission/processing section 10, an additional delay time to be additionally given to the audio data to be transmitted to the receiving side station device 200 is calculated.

The variable delay means 6 additionally provides the additional delay time to the audio data to be transmitted to the receiving station device 200.

be exposed.

[For production]

The additional delay time calculating means 3 obtains the delay time α in the image data transmitting side intra-office processing/transmission unit l by the transmitting side intra-office delay time detecting means 2, and calculates the delay time α in the image data receiving side intra-office transmission/transmission unit l.
The delay time α2 in the processing section 7 is obtained via the receiving side intra-station delay time detection means 9, the receiving side station delay information transmitting means 8, and the receiving side station delay information collecting means 4. In addition, the delay time β1 in the audio data transmitting side internal processing/transmission section 5 and the delay time β2 in the audio data receiving side internal transmission/processing section 10 are fixed data, so they are stored in advance and are applied to the audio data. The additional delay time to be additionally given to the delay time is determined as (α1+αt)−(β1+β2), and is set in the variable delay means 6.

In this way, adjustments are automatically made to ensure lip synchronization according to the mode of processing applied to image data and the processing capacity of the receiving device. [Embodiment] FIG. 2 shows an embodiment of the present invention. FIG.

The configuration shown in FIG. 2 transmits image data and audio data corresponding to the image data to the other station via the transmission path 32.
This figure shows the configuration of a terminal device that transmits and receives data in both directions.

In FIG. 2, 21 and 25 are selectors, 22 is an image data encoding circuit, 23 and 26 are buffer registers, 24 is an image data decoding circuit, 27 is a control unit, and 28
29 is an audio data encoding circuit, 29 is a variable delay circuit, 30 is an audio data decoding circuit, 31 is a transmission path interface, and 32 is a transmission path.

The image data encoding circuit 22 corresponds to the image data transmitting side intra-office processing/transmission section 1 in the configuration shown in FIG. Corresponds to section 7. Further, the audio data encoding circuit 28 corresponds to the audio data transmitting side intra-office processing/transmission unit 5 in the configuration shown in FIG. 1, and the audio data decoding circuit 30 corresponds to the audio data receiving side intra-office transmission - Corresponds to the processing unit 10. The variable delay circuit 29 corresponds to the variable delay means 6 in the configuration of FIG.

Here, in the configuration of FIG. 2, the transmitting side intra-office delay time detection means 2 and the receiving side intra-office delay time detection means 9 of FIG.
Before explaining the configuration of the part that realizes the function, referring to FIG. The procedure for detecting the delay time incurred will be explained.

As shown in FIG. 5, a data transmission frame consists of a frame header containing various control information and a data portion.Normally, the data transmission frame is transmitted to the image data encoding circuit 22 (or image data decoding circuit 22). (Similarly in the circuit 24), first, (1) the frame header is removed.

Next, the contents of the ① data portion are subjected to encoding processing to change D→D'. Then, ■ reattach the frame header that was removed at the beginning to the beginning.

By the way, in order to detect the delay time mentioned above, one pit for delay time detection is provided in the frame header.Normally, this bit is set to "0", and this bit is set only when detecting the delay time. “0” → “1”.

Then, on the output side of the image data encoding circuit 22 (or the same applies to the image data decoding circuit 24), the timing at which the delay time detection bit in the frame header of the output data transmission frame becomes "l". Detect.

Returning to the configuration shown in FIG. 2, the procedure shown in FIG. 5 described above is realized as follows.

First, when detecting the delay time in the image data encoding circuit 22, the control section 27 controls the selector 21 to
At the timing of the delay time detection bit in the frame header shown in FIG. , and its contents are detected.This realizes the function of the transmitting side intra-station delay time detection means 2 shown in FIG.

Similarly, when detecting the delay time in the image data decoding circuit 24, the control unit 27 controls the selector 25.
is controlled to change the bit from “0” to “1” at the timing of the delay time detection bit in the frame header shown in Figure 5.
Then, the output of the image data decoding circuit 24 is taken into the buffer circuit 26 and its contents are detected.

This realizes the function of the transmitting side intra-station delay time detection means 90 shown in FIG.

Note that the control section 27 in FIG. 2 is composed of a microcomputer equipped with an MPU.

The functions of the transmitting station delay information collecting means 4 and the transmitting station delay information transmitting means 8 in FIG.
This is done via the transmission line interface 31. The transmission line interface 31 has a function as a communication multiplexing device, and as described above, between the terminal device having the configuration shown in FIG. 2 and the opposite terminal device having a similar configuration. It multiplexes or separates image data and audio data that are transmitted bidirectionally, as well as control information that is exchanged between control units 27, for example, as will be described later.

Now, the procedure for exchanging delay information between two terminal stations is as follows.
As shown in FIGS. 3 and 4.

First, in FIG. 3, it is assumed that both "station A" and "station 18" have the configurations shown in FIG.

As shown in FIG. 3, when the devices are started up, they mutually inquire about the device capabilities of the other station (for example, the “B station” for the “A station”). This inquiry includes an inquiry about the delay time in the image data decoding circuit 24 in the other station.

When both the "A station" and the "B station" receive the above inquiry, the image data decoding circuit in the own station device uses the reception-side intra-station delay time detection function in the own station device, as described above. 24 is detected, and the controller 27 sends it to the other station via the transmission line interface 31 as an "apparatus capability response."

The transmission frame at this time, as shown in FIG.
The delay time information in No. 4 is transmitted as part of the control information.

In the device at the other station, the transmission line interface 3
1, the control unit 27 receives the transmission frame,
Recognize the delay time mentioned above.

Then, the control units 27 of both the A and B stations respectively control the delay time in the image data encoding circuit 22 in the own station device, and also transmit the audio data stored in advance in the control unit 27. Based on the delay time data in the side station internal processing/transmission section 5 and the audio data receiving side intra-station transmission/processing section 10, the calculations as shown in the [Operation] column were performed earlier to obtain the value. is set in the variable delay circuit 29 in FIG.

When the settings are completed, a RAS signal is transmitted, negotiation is completed, and data communication is started.

Note that FIG. 7 shows an example of the configuration of the variable delay circuit 29.

In FIG. 7, 41 and 42 are flip/70-tub circuits, 43 is a control clock generation circuit, 44 is a RAM, 45 is a counter, 46 is a register, and 47 is a frequency dividing circuit.

The input to the variable delay circuit 29, ie, the output of the audio data encoding circuit 28 having the configuration shown in FIG. 2, is applied to the flip-flop circuit 41 shown in FIG.

A control clock outputted from the control clock generation circuit 43 and synchronized with the timing of the output of the audio data encoding circuit 2 is generated by the flip-flop circuits 4I and 4.
It is applied as an edge trigger input of 2, and after being frequency divided by 1/2 via the frequency divider circuit 47, it is applied as a manual R/W for read/write control of the RAM 44, and as an edge trigger input of the counter 45. is being applied.

The aforementioned delay time value from the control unit 27 in FIG. The count output Q of the counter 45 is applied as the address of the RAM 44, and the ripple carry output RC of the counter 45 is applied to the load control input. Further, the counter 45 is a down counter, and with the above configuration, the count output Q of the counter 45 circulates at the cycle of the delay time value set in the register 46.

Further, the output of the audio data encoding circuit 28 is serially applied to the data input terminal DATA-IN of the RAM 44 via the flip-flop circuit 4I.

In this way, for example, when the output of the frequency dividing circuit 47 is at H level, the RAM 44 uses this as a read signal, and stores the above-mentioned data one cycle before the delay time value stored in the address at that time. The output of the audio data encoding circuit 28 is read. The read output is sent to the flip-flop circuit 42 at the next rising edge of the control clock.
is input. Thus, the flip-flop circuit 42
The output of the audio data encoding circuit 28 one cycle before the value of the delay time appears on the output side.

Next, when the output of the frequency dividing circuit 47 changes from H to L, the output of the frequency dividing circuit 47 becomes a write signal, but at this time, the address of the RAM 44 has not been updated yet, so the delay time described above is The new output of the audio data encoding circuit 28 is written to the address from which the data one cycle before the value was read.

Thereafter, when the output of the frequency dividing circuit 47 changes from L to H again, the counter 45 is decremented as described above, and the value of the delay time stored in the adjacent (decremented) address is The data one cycle before is read out in the same manner as described above. Further, when the output of the frequency dividing circuit 47 changes from H to L, the adjacent (decremented)
A new output (next bit) of the audio data encoding circuit 28 is written to the address.

The same operation is repeated thereafter. In this way, with the configuration shown in FIG. 7, the delay of the above-mentioned delay time is given to the output of the audio data encoding circuit 2B.

〔Effect of the invention〕

According to the present invention, it is possible to automatically make adjustments to ensure lip synchronization according to the mode of processing applied to image data in the sending device and the receiving device, and the processing capacity of the receiving device. Make it.

[Brief explanation of the drawing]

Fig. 1 is a basic configuration diagram of the present invention, Fig. 2 is a configuration diagram of an embodiment of the invention, Fig. 3 is a diagram showing the procedure for inquiring about equipment capabilities and responding, and Fig. 4 is a diagram showing equipment startup. 5 is a diagram showing an example of the procedure for detecting delay time, FIG. 6 is a schematic configuration diagram of a conventional image/audio transmission system, and FIG. 1 is a diagram illustrating a configuration example of a variable delay circuit. [Explanation of symbols] 1... Image data transmitting side internal processing/transmission unit, 2...
Transmitting side intra-station delay time detection means, 3... Additional delay time calculating means, 4... Transmitting side delay information collecting means, 5...
Audio data transmitting side intra-office processing/transmission unit, 6... variable delay means, 7... image data receiving side intra-office transmission/processing unit, 8
... Transmission station delay information transmission means, 9... Receiving side intra-station delay time detection means, IO... Audio data receiving side intra-station transmission/processing section, 11+, 1-1□, lls, 32°66,
．． 66 O...Transmission line, 21.25...Selector, 2
2... Image data encoding circuit, 23.26... Bar.
Sofa register, 24...image data decoding circuit, 2
7... Control unit, 28... Audio data encoding circuit, 2
9... Variable delay circuit, 30... Audio data decoding circuit, 31... Transmission line interface, 32... Transmission line, 41.42... Flip-flop circuit, 43
...Control clock generation circuit, 44...RAM, 45
...Counter, 46...Register, 61...Image data processing section, 62...Transmission path within the image data sending side station, 63...Audio data processing section, 65...Inside the audio data sending side station Transmission line, 64... variable delay means, 67.
... Image data receiving side intra-office transmission line, 68... Image data processing section, 69... Audio data receiving side intra-office transmission line, 7
0... Audio data processing section, 100, 101... Transmitting side station device, 200.201... Receiving side station device.

Claims (1)

[Scope of Claims] 1. An image data transmitting station internal processing/transmission unit (1) that applies predetermined processing to image data to be transmitted and transmits it to a receiving station (200); Transmission comprising an audio data transmitting side internal processing/transmission unit (5) that applies predetermined processing to the audio data to be transmitted to the receiving side station (200) and transmits the same to the receiving side station (200). Side station equipment (
100), an image data receiving side intra-office transmission/processing unit (7) that applies predetermined processing to the image data transmitted from the transmitting side station device (100) and outputs the resultant image data, and In an image/audio transmission system comprising a receiving side station device (200) comprising an audio data receiving side intra-office transmission/processing unit (10) that applies predetermined processing to audio data transmitted by the user and outputs the resultant data, The receiving side station device (200) includes a receiving side intra-station delay time detection means (9) for detecting a delay time in the image data receiving side intra-station transmission/processing section (7).
and receiving side station delay information transmitting means (8) for transmitting the delay time in the image data receiving side intra-station transmission/processing unit (7) to the transmitting side station device (100), the transmitting side station device (100) includes a receiving side station delay information collecting means (4) for collecting delay time in the image data receiving side intra-station transmission/processing unit (7) from the receiving side station device (200); and the image data transmitting unit. Transmitting side station delay time detection means (2) for detecting delay time in the side station processing/transmission unit (1)
The delay time in the audio data transmitting side intra-station processing/transmission section (5) and the delay time in the audio data receiving side intra-station transmission/processing section (10) are stored in advance as fixed data, and The delay time in the processing/transmission section (5) and the audio data receiving side intra-office transmission/processing section (10), and the image data sending side intra-office processing/transmission section (
From the delay time in 1) and the delay time in the image data receiving station internal transmission/processing unit (7) collected by the receiving station delay information collecting means (4), the receiving station device (200) additional delay time calculation means (3) for calculating an additional delay time to be additionally given to the audio data to be transmitted to the receiving station device (200); An image/audio transmission system comprising: variable delay means (6) for additionally providing the additional delay time.