JP2005151584A - Transmission processing apparatus and method - Google Patents

Abstract

An object of the present invention is to eliminate the need to provide video / audio encoding means in each communication terminal.
A switch 12 selects one from a plurality of video / audio inputs. The video capture 16 of the transmission processing device 14 digitizes the selected video signal. The video encoder 18 converts the output of the video capture 16 into motion JPEG, MPEG and H.264. Compress and encode according to H.261. The selector 20 selects the compressed video data to be transmitted from the compressed video data of each encoding method, and supplies it to the communication buffer 22. The voice encoder 24 digitizes and encodes the selected voice signal, and supplies it to the communication buffer 22. The encoded video data and encoded audio data in the communication buffer 22 are read out to the network 30. The control circuit 26 controls the devices 12 to 24 according to the control command received via the network 30.
[Selection] Figure 1

Description

  The present invention relates to a transmission processing apparatus and method for selectively outputting video / audio signals from a plurality of video / audio sources to a network, and more specifically to a transmission processing apparatus and method corresponding to a computer network.

  The configuration of a communication terminal device in which video and audio input / output devices such as a camera, video monitor, microphone and speaker are connected to a computer (personal computer or workstation) so that video and audio can be transmitted and received between terminals is well known. For example, it is already generally used as a communication terminal device for a video conference or a video conference.

  The video signal input from the camera and the audio signal input from the microphone are each digitized, encoded by a predetermined method, and output to a network such as a local area network or a wide area network. The encoded video signal and audio signal from the network are decoded and output from the video monitor and the speaker, respectively. The microphone and the speaker are combined with an echo canceller for avoiding an echo caused by a sound wraparound. Some are configured as speakerphones.

  Conventionally, such a video conference system has been configured by a dedicated communication terminal. However, by improving the processing capability of the computer, a video conference or a video conference can be performed using a computer on the desktop of each person in the office as a communication terminal. It has become to.

  In applications where video and / or audio are transmitted to remote locations, there are not only video conferences but also remote monitoring systems for observing various situations.

  In video transmission, a video capture device for digitizing and capturing a video signal output from a camera and a video compression device for compression coding are expensive. When equipped on each person's computer, the cost-effectiveness ratio is low in view of its availability.

  In applications such as a remote monitoring system, it is not always necessary to update a single point image at such a high frame rate with respect to the performance of the video compression apparatus (for example, encoding of 30 frames per second). For example, there is a usage form in which one frame per second is sufficient. In such a mode of use, a video compression device capable of 30 frames per second is over-specified.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a transmission processing apparatus and method having a simple configuration and a good cost-effectiveness ratio.

  A transmission processing apparatus according to the present invention is a transmission processing apparatus that selects one of a plurality of inputs for at least one of video and audio, and transmits the selected one to a network. Switching means for switching in response, an A / D conversion means for digitizing an analog signal output from the switching means, an encoding means for compressing and encoding the digital output of the A / D conversion means, and the encoding means Output means for outputting the data encoded by the above to a network.

  The transmission processing device according to the present invention is a transmission processing device that selects one of a plurality of inputs for video and audio and transmits the selected one to a network, and a switching means for switching the plurality of inputs via the network. And an encoding means for compressing and encoding the signal from the switching means, and an output means for outputting the output encoded by the means to the network.

  A transmission processing apparatus according to the present invention includes a switch that selects an arbitrary plurality of signals from a plurality of analog input signals, and a combining unit that compresses and combines a plurality of analog signals selected by the switch on a time axis, A / D conversion means for digitizing the analog signal output from the synthesizing means, encoding means for compressing and encoding the digital signal output from the A / D conversion means, and control for controlling the communication and the switch Means.

  A transmission processing apparatus according to the present invention includes a switch for selecting an arbitrary plurality of signals from a plurality of analog input signals, and a plurality of A / D conversion means for digitizing each of the plurality of analog signals selected by the switch. The digital signal output from the plurality of A / D conversion means includes a plurality of encoding means for compressing and encoding, and a control means for controlling the communication and the switch.

  A transmission processing method according to the present invention is a transmission processing method of selecting one of a plurality of inputs for at least one of video and audio, and transmitting the selected one to a network. Switching is made accordingly, digitizing by A / D conversion means, the digital output is compression encoded, and the encoded data is output to the network.

  A transmission processing method according to the present invention is a transmission processing method for selecting one of a plurality of inputs for video and audio and transmitting the selected one to a network. The transmission processing method switches between a plurality of inputs via the network and performs compression. It encodes and outputs an encoding output to a network.

  The transmission processing method according to the present invention selects an arbitrary plurality of signals from a plurality of analog input signals, compresses and combines the selected analog signals on the time axis, and digitizes the combined analog signals. The digitally synthesized signal is compression-encoded.

  The transmission processing method according to the present invention is characterized by selecting a plurality of arbitrary signals from a plurality of analog input signals, digitizing each of the selected plurality of analog signals, and compressing and encoding each digital signal. .

  According to the present invention, resources for video / audio information transmission processing can be shared by a plurality of terminals, the operating rate of these resources can be increased, and the cost-effectiveness ratio can be improved.

  By providing a plurality of compression encoding means and selecting the compression encoding means based on the information exchange with the communication destination and / or the state of the communication path, the video can be transmitted with an appropriate communication quality according to the performance and situation of the transmission destination. / Voice information can be transmitted.

  By incorporating it into a video conference system, a video conference system can be constructed at a lower cost. By concentrating the video / audio encoding means, the video conference system terminal can be made inexpensive.

  In addition, according to the present invention, by using a switch that selects a plurality of arbitrary signals from a plurality of input signals, the capture device and the encoder can be shared, and the operating rate of these modules is increased and the cost effectiveness ratio is increased. Can be improved. In addition, by simultaneously digitizing and compressing and encoding a plurality of input signals, time loss due to switch switching can be reduced.

  Further, according to the present invention, since the compression encoding means is selected based on the information exchange with the communication destination and the status of the communication path, information can be transmitted with appropriate communication quality according to the performance and status of the transmission destination.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of a first embodiment of the present invention. The video / audio transmission processing apparatus 10 of this embodiment includes a switch 12 that selects video / audio signals to be transmitted from a plurality of (four in this embodiment) video / audio inputs, and the video / audio selected by the switch 12. And a transmission processing device 14 for transmitting a signal.

  The video capture 16 in the transmission processing device 14 digitizes the video signal selected by the switch 12. Video encoder 18 includes motion JPEG, MPEG and H.264. And an encoding module corresponding to each encoding method of H.261, and an output signal of the video capture 16 is converted into motion JPEG, MPEG and H.264. Compress and encode according to H.261. The selector 20 selects compressed video data to be transmitted from the compressed video data of each encoding method, and supplies the selected data to the communication buffer 22. On the other hand, the audio encoder 24 digitizes and encodes the audio signal selected by the switch 12 and supplies it to the communication buffer 22. The encoded video data and encoded audio data written in the communication buffer 22 are read out to the network 30 at an appropriate format and speed.

  The control circuit 26 controls the switch 12, the video capture 16, the video encoder 18, the selector 20, and the audio encoder 24 according to the control command received via the network 30.

  FIG. 2 is a flowchart showing the operation of the embodiment shown in FIG. First, it waits for events such as a transmission request, a transmission end request, and transmission activation (S1). When a video / audio transmission request is received (S2), a new transmission process is started (S3). When a video / audio transmission end request is received (S4), the designated video / audio transmission processing is ended (S5). When an event for starting the video / audio transmission operation is generated by the transmission scheduler (S6), acquisition of image data from the camera and transmission processing are started (S7).

  In this embodiment, there are two types of transmission requests. The first is a case where high frame rate video data communication is requested as in a video conference. In this case, when a frame rate exceeding the switching speed of the switch 12 or the transmission processing device 14 is requested, it is necessary to occupy all transmission operations in order to meet the request. Such a request is called an exclusive processing request. The second is a case where a high frame rate is not required as in monitoring while switching cameras. Such a request will be referred to as a non-occupied processing request.

  A plurality of non-occupying processing requests can be accepted, and the transmission scheduler sequentially selects and transmits signals from a designated video / audio source. The transmission scheduler is actually realized by a timer event.

  First, the control circuit 26 enters an event waiting state (S1). When an external communication or timer event of the transmission scheduler occurs, the process proceeds to S2, S4, or S6 depending on the communication content and the type of event.

  When the transmitted message is a new transmission request, the process proceeds to a video / audio transmission request acceptance process (S2), and the content of the message is read. Next, a transmission start process (S3) is executed according to the message content.

  If the transmitted message is a transmission end request, the process proceeds to an audio / video transmission end request acceptance process (S4), and the content of the message is read. Next, a transmission end process (S5) is executed according to the content of the message.

  If a timer event occurs, the transmission process is restarted and the process returns to the event waiting state (S1). The started transmission processing process acquires and transmits video data from the camera, and sets a timer (S7).

  FIG. 3 shows a detailed flowchart of the transmission start process (S3) of FIG. First, the control circuit 26 determines whether or not an exclusive processing request has already been made (S11). If there is an exclusive processing request (S11), since no other request can be accepted, the start request is rejected (S15). When there is no exclusive process request (S11), it is determined whether the process itself to be started is an exclusive process request (S12). In the case of a non-occupied process request (S12), a transmission request process is added to the transmission scheduler, a timer for activation is set, and the process ends (S13). In the case of the exclusive processing request (S12), since it cannot coexist with other requests, the processing process of the existing request is temporarily stopped (S14), and then the transmission request processing process is added to the transmission scheduler, A timer is set and the process ends (S13).

  FIG. 4 shows a detailed flowchart of the transmission end process (S5) of FIG. First, the control circuit 26 determines whether or not the designated process to be ended actually exists with reference to the transmission scheduler (S21). If there is no process designated to end (S21), an error message is returned (S24). If there is an end-designated process (S21), an end signal is sent to the corresponding transmission request processing process, and the process is deleted from the transmission scheduler (S22). Thereafter, it is determined whether the terminated process is due to an exclusive processing request (S23). If it is due to an exclusive processing request (S23), the process suspended by the process is referred to the transmission scheduler. Then, the system is returned to the restartable state, and the process ends (S25).

  Acquisition of video data from the camera and the content of the transmission process (S7) differ depending on whether or not the processing request is an exclusive processing request. FIG. 5 shows the details of S7 of FIG. 2 and shows the flow of processing for the non-occupied processing request. First, the control circuit 26 sets the switch 12 so as to input the requested camera image (S31), and sets the video capture 16 so as to obtain the requested image size (S32). The selector 20 is set so as to select the encoded data corresponding to the requested compression method among the three outputs of 18 (S33), and the encoding parameter of the video encoder 18 is set (S34).

  Thereafter, the camera video is captured and data is transmitted. That is, the video capture 16 digitizes the video signal from the switch 12 (S35), and the video encoder 18 compresses and encodes the digitized video signal (S36). The encoded video data of the specified encoding method is selected by the selector 20 and stored in the communication buffer 22 (S37). At the same time, the audio encoder 24 digitizes and encodes the selected audio signal and stores it in the communication buffer 22. The encoded video data and encoded audio data stored in the communication buffer 22 are read to the network 30 at an appropriate rate and format and transmitted to the request source (or designated partner) (S38). After transmission, the timer is set to the next transmission time and the process ends.

  In this embodiment, encoding processing is performed in all three encoding modules in the video encoder 18, but only the one corresponding to the selected encoding method may be operated.

  6 and 7 show details of S7 in FIG. 2 and show the flow of processing for the exclusive processing request. FIG. 6 shows the initialization process, and FIG. 7 shows the flow of the video data acquisition and transmission process from the camera. In the case of the exclusive processing request, the initialization setting operation corresponding to S31 to S34 of the non-occlusive processing request is performed only for the first time (and when the transmission request parameter is changed), and thereafter, only the data acquisition and transmission operations are repeated. It is.

  As an initialization process, as shown in FIG. 6, first, the control circuit 26 sets the switch 12 to input the requested camera image (S41), and the video / video size is set to the requested image size. When the capture 16 is set (S42), the selector 20 is set so as to select the encoded data corresponding to the requested compression method among the three outputs of the video encoder 18 (S43). An encoding parameter is set (S44). These processes are the same as S31 to S34 in FIG.

  After these initialization processes, the camera image acquisition and transmission processes are actually repeated. That is, the video capture 16 digitizes the video signal from the switch 12 (S51), and the video encoder 18 compresses and encodes the digitized video signal (S52). The encoded video data of the designated encoding method is selected by the selector 20 and stored in the communication buffer 22 (S53). At the same time, the audio encoder 24 digitizes and encodes the selected audio signal and stores it in the communication buffer 22. The encoded video data and encoded audio data stored in the communication buffer 22 are read out to the network 30 at an appropriate rate and format and transmitted to the request source (or designated partner) (S54). After transmission, the timer is set to the next transmission time and the process ends.

  In the above description, the operation of sending an input signal from one camera or microphone to one transmission destination has been shown. However, when there are transmission requests for input signals from one camera or microphone from a plurality of terminals, It is obvious as a network technology that data can be sent to a plurality of destinations at once.

  When there is a request to change parameters such as the size and rate of transmission data, the initialization process of FIG. 6 is re-executed, and then the operation of FIG. 7 continues. At this time, if the exclusive processing request becomes a non-occupying processing request due to a parameter change, the stopped processing process is returned to a restartable state.

  Also, the steps (S31 to S37 in FIG. 5 and S51 to S53 in FIG. 7) until the encoding of the video / audio data and the storage in the communication buffer 22 and the transmission of the data stored in the communication buffer 22 (in FIG. 5). S38 and S54 in FIG. 7 are made separate processes (encoding process and transmission process) and two buffers are prepared, and the transmission process executing S38 or S54 outputs the data of one buffer to the network. In the meantime, the data transmission efficiency can be improved by allowing the encoding process to store the next data to be transmitted in the other buffer. Such a technique itself is known as a parallel processing technique.

  FIG. 8 shows a schematic diagram of the network configuration of the video conference system using the embodiment shown in FIG. The camera output and microphone output of the two terminals 40a and 40b are input to the switch 12 of the video transmission processing apparatus 10 shown in FIG. The terminal 40a is based on a workstation 42a, and the terminal 40b is based on a personal computer 42b, to which cameras 44a and 44b, microphones 46a and 46b, video monitors 48a and 48b, and speakers 50a and 50b are connected, respectively. The microphones 46a and 46b and the speakers 50a and 50b have a speaker phone configuration, for example.

  As an example, the terminal 40a is connected to the network 30 via a high-speed network such as FDDI, and the workstation 42a includes hardware for decoding a video / audio encoded signal, for example. The terminal 40b is connected to the network 30 via the Ethernet 52 and the router 54, and the computer 42b includes software for decoding a video / audio encoded signal. Here, it is assumed that the network 30 to which the output of the video transmission processing device 10 is connected is a backbone network or a high-speed network close thereto.

  A state where a video conference is held in this environment will be described. It is assumed that a video conference is performed between the terminal 40a and another terminal (not shown).

  In a video conference, the user's image and the image of a communication partner (or a plurality of persons) are displayed on a monitor, and the other's voice is output from the speaker of his / her terminal. When the terminal 40a is the own terminal, this is realized by starting the reception processing of the own terminal video and starting the reception processing of the video and audio of the partner terminal.

  Since the reception processing of the own terminal video and the reception processing of the video and audio of the counterpart terminal are almost the same operation, the video and audio reception processing of the counterpart terminal will be described below as an example.

  FIG. 9 shows a video / audio reception process that operates on its own terminal, and a data transmission process of a video / audio transmission processing apparatus (hereinafter referred to as a counterpart video / audio transmission processing apparatus) to which the camera output and microphone output of the other party are input. The relationship is shown. S61 to S67 indicate video / audio reception processing operating on the own terminal, and S68 to S70 indicate data transmission processing of the counterpart video / audio transmission processing device.

  First, reception processing is started on the own terminal. Modules on the window system necessary for video display and audio output are initialized (S61), and a buffer for receiving data is prepared (S62). A port for reception is opened, and reception is enabled (S63). The other party's video / audio transmission processing device is requested to transmit the camera output and microphone output of the other terminal (S64). Upon receiving this request, the counterpart video / audio transmission processing device, if accepted, executes initial settings for transmission and requests a communication connection to the reception port on the counterpart terminal (terminal 40a) side (S68). As a result, the terminal itself (terminal 40a) establishes a communication connection with the counterpart video / audio transmission processing apparatus (S65).

  The partner video / audio transmission processing apparatus acquires and encodes the video and audio data, stores them in the communication buffer (S69), and transmits the data stored in the communication buffer to the communication partner (terminal 40a) (S70).

  The own terminal (terminal 40a) receives the encoded data (S66), decodes it, displays the video on the window of the video monitor 48a, and outputs the sound from the speaker 50a (S67).

  Thereafter, the partner video / audio transmission processing apparatus repeats S69 and S70, and the terminal 40a also repeats S66 and S67. As a result, video and audio are continuously transferred and reproduced.

  The video output and the microphone output of the camera may be branched at an analog signal level and input to both the computers 42a and 42b and the video / audio transmission processing device 10. In this case, it is not necessary to receive own video, but the video capture function for digitizing the camera input is required for the computers 42a and 42b.

  Note that the burden of wiring can be eliminated by wirelessly transmitting the outputs of the cameras 44a and 44b and the microphones 46a and 46b to the video / audio transmission processing apparatus 10.

  In this way, an environment for transmitting video / audio information of a large number of cameras and microphones can be realized at low cost. Also, video / audio data can be transmitted / received using different compression encoding methods and appropriate transmission parameters according to the performance and function of the receiving terminal.

  The audio signal processing may be omitted, and only the video signal may be transmitted. FIG. 10 shows a schematic block diagram of a video transmission processing apparatus in which audio signal processing is omitted. From the embodiment shown in FIG. 1, the voice input to the switch 12 is eliminated, and the voice decoder 24 of the transmission processing device 14 is removed. For applications such as monitoring using video, audio information may not be necessary, and by configuring as shown in FIG. 10, a system with a better cost-effectiveness ratio can be obtained. FIG. 11 is a configuration example of a remote monitoring system using the video transmission processing device shown in FIG. 10, and is obtained by removing the voice input / output device from the configuration shown in FIG.

  By configuring in this way, it is possible to configure a system that is optimal for a purpose such as monitoring at a low cost with a cheaper configuration.

  It is clear that the configuration shown in FIG. 8 can be extended to a wide area network (WAN). For example, as shown in FIG. 12, a video / audio WAN switch 60 generally called a video gateway or the like may be combined.

  With this configuration, data can be exchanged with an external network via a public communication line network such as ISDN.

  In the above-described embodiment, by sharing the video capture for capturing the video signal and the video and audio encoder, the cost for that can be reduced, but generally, the switch 12 cannot use a very high speed (expensive). Therefore, in applications that require a high frame rate, a plurality of signals cannot be handled substantially simultaneously.

  An embodiment that solves this problem will be described below. FIG. 13 shows a schematic block diagram of a third embodiment of the present invention. 110 is a video / audio transmission server, 112 is a matrix switch that can arbitrarily select four video / audio signals from eight video / audio inputs, and can be arbitrarily output from four output ports of the selected four video / audio signals. , One of the four video / audio signals from the matrix switch 112 is selected, or the four video / audio signals are time-axis-compressed, and the video signal is synthesized into a multi-screen configuration of four screens. Is a screen dividing unit that compresses and synthesizes to 1/4 on the time axis.

  The video / audio transmission server 110 includes a video capture device 116 that captures and digitizes a video signal output from the screen division unit 114, a video encoder 118 that compresses and encodes output video data of the video capture device 116, and a screen division unit. 114, an audio capture device 120 that captures and digitizes an audio signal output from 114, an audio encoder 122 that compresses and encodes audio data output from the audio capture device 120, encoded data of encoders 118 and 122, and information to be transmitted Is composed of a communication buffer 124 for temporarily storing the video and audio transmission server 110, a matrix switch 112, and a control circuit 126 for controlling the screen division unit 114.

  As shown in FIG. 14, the screen division unit 114 has two operation modes: a division mode and a selection mode. The division mode is a mode in which four input images are reduced in half horizontally and vertically and combined into one screen. At this time, the screen division unit 114 compresses and synthesizes the four input sounds on the time axis and outputs them. The selection mode is a mode for selecting and outputting any one of the four input video / audio signals. As an apparatus having such a function, for example, there is YS-Q430 manufactured by Sony Corporation.

  For example, as shown in FIG. 15, the encoder 118 includes a plurality of encoders 130, 132, and 134 that correspond to different compression encoding methods (for example, Motion JPEG, MPEG, and ITU-T recommendation H.261), and a control circuit The switch 136 may be configured to supply the input video data to any one of the designated encoders 130, 132, and 134 under the control of 126. The same applies to the voice encoder 122.

  The control circuit 126 sets the matrix switch 112 and the screen division unit 114 according to the content of the transmission request from the outside, and controls the video capture device 116, the video encoder 118, the audio capture device 120, and the audio encoder 122. Appropriate transmission data is generated and transmitted to the external network via the communication buffer 124.

  FIG. 16 is a flowchart showing the operation of the embodiment of FIG. First, it waits for events such as a transmission request, a transmission end request, and transmission activation (S101). When a video / audio transmission request is received (S102), a new transmission process is started (S103). When the video / audio transmission end request is received (S104), the designated video / audio transmission processing is ended (S105). When an event for starting the video / audio transmission operation is generated by the transmission scheduler (S106), acquisition of video / audio from the camera / microphone and transmission processing are started (S107).

  Also in this embodiment, there are two types of transmission requests: exclusive processing requests and non-occupying processing requests. The exclusive processing request is a transmission request that enables data communication at a high frame rate by occupying all transmission operations with specific data. When the frame rate requested to the video / audio transmission server 110 exceeds the switching speed of the matrix switch 112, it is necessary to make an exclusive processing request. The non-occupying processing request is a transmission request for transmitting a plurality of video / audio signals without assigning a high frame rate. A plurality of non-occupied processing requests can be accepted and are sequentially activated by the transmission scheduler. The transmission scheduler is actually realized by a timer event.

  Each transmission request is further divided into two types. The first is a data transmission request in which a video frame rate is given priority over resolution in order to perform a video conference or the like. At this time, the operation mode of the screen division unit 114 is set to the division mode, and a plurality of screens are fetched without switching the matrix switch 112 and encoded. The second is a data transmission request that gives priority to resolution over frame rate in order to observe details of an object. At this time, the operation mode of the screen division unit 114 is set to the selection mode, and the video / audio signal to be transmitted is selected by the matrix switch 112.

  First, the control circuit 126 enters an event waiting state (S101). When an external communication or transmission scheduler timer event occurs, the process proceeds to S102, S104, or S106 depending on the communication content and the type of event.

  If the received message is a new transmission request, the process proceeds to a video / audio transmission request acceptance process (S102), and the content of the message is read. Next, a transmission start process (S103) is executed according to the message content.

  If the received message is a transmission end request, the process proceeds to an audio / video transmission end request acceptance process (S104), and the content of the message is read. Next, transmission end processing (S105) is executed according to the content of the message.

  If a timer event has occurred, the transmission process is restarted and the process returns to the event waiting state (S101). The activated transmission process executes the process of acquiring and transmitting video / audio from the camera / microphone, and sets the timer again (S107).

  FIG. 17 shows a detailed flowchart of the transmission start process S103 of FIG. First, the control circuit 126 determines whether or not an exclusive processing request has already been made (S111). When there is an exclusive processing request (S111), since no other request can be accepted, the start request is rejected (S115). When there is no exclusive process request (S111), it is determined whether the process to be started is an exclusive process request (S112). In the case of a non-occupied processing request (S112), a transmission request process corresponding to resolution priority or frame rate priority is added to the transmission scheduler, a timer for activation is set, and the process ends (S113). In the case of the exclusive processing request (S112), since it cannot coexist with other requests, the processing process of the existing request is suspended (S114), and then the transmission request processing process is added to the transmission scheduler to The timer is set and the process ends (S113).

  FIG. 18 shows a procedure for adding a non-occupied processing request processing process to the transmission scheduler. First, a processing request is received (S121), and it is checked whether a transmission processing request associated with the same camera as that requested by the received processing request already exists in the transmission scheduler (S122). If there is a transmission processing request for the same camera (S122), the transmission destination of the transmission request just received is added to the transmission processing process, and the process ends (S123). If there is no transmission processing request for the same camera (S122), it is determined whether the processing request received in S121 is resolution priority or frame rate priority (S124). When the resolution is prioritized (S124), a new timer for activation is set and the process ends (S125). If the frame rate is prioritized (S124), it is checked with reference to the transmission scheduler whether or not there is a space in the frame rate priority processing request queue (S126). When the timer for starting is set and finished (S8), if there is a vacancy (S126), processing is added to the vacant place and the process is finished (S8127).

  With this procedure, the transmission schedule of the transmission scheduler is as shown in FIG. 20 for the transmission request sequence illustrated in FIG.

  FIG. 21 is a detailed flowchart of the transmission end process S105 of FIG. First, the control circuit 126 determines whether or not the designated process to be ended actually exists with reference to the transmission scheduler (S131). If there is no process designated to end (S131), an error message is returned (S134). If there is an end-designated process (S131), an end signal is sent to the corresponding transmission request processing process, and the process is deleted from the transmission scheduler (S132). Thereafter, it is determined whether the terminated process is due to an exclusive processing request (S133). If it is due to an exclusive processing request (S133), the process suspended by the process is referred to the transmission scheduler. Then, the system is returned to the restartable state, and the process ends (S135).

  The contents of the processing of S107 in FIG. 16, that is, acquisition of video / audio data from the camera / microphone and transmission processing differ depending on whether or not the processing request is an exclusive processing request.

  FIG. 22 is a detailed flowchart of S107 in FIG. 16 and shows a flowchart in the case of a non-occupying process request. First, the control circuit 126 sets the operation mode of the screen division unit 114 to the selection mode if the request is resolution priority, and to the division mode if the request is frame rate priority (S141). The matrix switch 112 is set so as to capture the requested video / audio of the camera / microphone (S142), and the parameters of the video capture device 116 and the audio capture device 120 are set so as to obtain the requested video size and sound quality. (S143). The video capture device 116 and the audio capture device 120 capture and digitize the video / audio data (S144).

  Compression encoding parameters (parameters such as encoding method and compression rate, and encoding range) are set in the video encoder 118 and the audio encoder 122 (S145), and the output of the capture devices 116 and 120 is compressed under the conditions. (S146). The encoding range of the compression encoding condition is, for example, the entire screen when the transmission request is resolution priority, or the screen divided by the screen division unit 114 when the frame rate priority is still encoded. It is one of the areas that are not. The encoded video / audio data is temporarily stored in the communication buffer 124 (S147), and is transmitted from the communication buffer 124 to the request source (or designated partner) via the network. After this transmission, the next transmission request processing process is added to the transmission scheduler. S145 to S148 are repeated until all captured areas have been transmitted (S149).

  Since an encoding parameter is set for each input and for each divided area (S145), each area can be encoded under different encoding conditions. Further, when the encoders 118 and 122 are compatible with a plurality of compression encoding systems, it is possible to encode with different compression encoding systems for each input.

  FIG. 23 and FIG. 24 show the details of S107 in FIG. 16 and show the flow in the case of an exclusive processing request. FIG. 23 shows the initialization process, and FIG. 24 shows the flow of the acquisition / transmission process of video / audio data from the camera / microphone. In the case of an exclusive processing request, the initialization setting operation corresponding to S141 to S143 and S145 of FIG. 22 is performed only for the first time (and when the transmission request parameter is changed), and thereafter, only data acquisition and transmission operations are repeated. It is.

  FIG. 23 will be described. First, the screen division unit 114 is set to the selection mode (S151). By setting in this way, a signal from a specific camera can be transmitted at high speed. Next, the matrix switch 112 is set so that the requested video / audio of the camera / microphone is input (S152). Parameters according to the content of the transmission request are set in the video capture device 116 and the audio capture device 120 (S153), and parameters according to the content of the transmission request are also set in the video encoder 118 and the audio encoder 122 (S122). Note that if the screen division unit 114 is set to the division mode in S151, videos from a plurality of cameras (up to four places in the example of FIG. 14) can be transmitted at high speed.

  FIG. 24 will be described. The video capture device 116 and the audio capture device 120 each capture and digitize the video signal and the audio signal output from the screen division unit 114 (S161). The video encoder 118 and the audio encoder 122 compress and encode the output data of the video capture device 116 and the audio capture device 120, respectively (S162), and store the encoded data in the communication buffer 124 (S163). The data stored in the communication buffer 124 is transmitted to the transmission request source (or designated partner) via the network at a predetermined rate and format (S164). After transmission, the timer is set to the next transmission time and the process ends.

  The steps of encoding video / audio data and storing them in the communication buffer 124 (S145 to S147 and S161 to S163) in the transmission processing process shown in FIGS. 22 and 24, and transmission of data stored in the communication buffer 124 ( S148 and S164) are separate processes and two buffers are prepared. While the transmission process executing S148 and S164 is transferring the data of one buffer, the encoding process should transmit the next data Can be stored in the other buffer, and writing and reading of the communication buffer can be executed simultaneously to improve the efficiency of data transmission.

  FIG. 25 shows a schematic diagram of the network configuration of the video conference system using the embodiment shown in FIG. Reference numeral 140 denotes a video / audio transmission processing apparatus including the video / audio transmission server 110, the matrix switch 112, and the screen division unit 114 shown in FIG. 13, and includes a local area network (LAN), a wide area network, and a general public telephone. It is connected to a communication network 142 including a line and other networks.

  The input of the matrix switch 112 of the video / audio transmission processing device 140 is connected to the camera output and microphone output of two terminals 144a and 144b arranged on a personal desk or side. The terminal 144a is based on the workstation 146a, and the terminal 144b is based on the personal computer 146b, to which cameras 148a and 148b, microphones 150a and 150b, video monitors 152a and 152b, and speakers 154a and 154b are connected, respectively. The microphones 150a and 150b and the speakers 154a and 154b have a speaker phone configuration, for example.

  As an example, the terminal 144a is connected to the network 142 via a high-speed network such as FDDI, and the workstation 146a includes hardware for decoding a video / audio encoded signal, for example. The terminal 142b is connected to the network 142 via the Ethernet (registered trademark) 156 and the router 158, and the computer 146b includes software for decoding a video / audio encoded signal. Here, it is assumed that the network 142 to which the output of the video / audio transmission processing device 140 is connected is a backbone network or a high-speed network close thereto.

  A state where a video conference is held in this environment will be described. It is assumed that a video conference is performed between the terminal 144a and another terminal (not shown).

  In a video conference, the user's image and the image of a communication partner (or a plurality of persons) are displayed on a monitor, and the other's voice is output from the speaker of his / her terminal. When the terminal 144a is the own terminal, this is realized by starting the reception processing of the own terminal video and starting the reception processing of the video and audio of the partner terminal.

  Since the reception processing of the own terminal video and the reception processing of the video and audio of the counterpart terminal are almost the same operation, the video and audio reception processing of the counterpart terminal will be described below as an example.

  FIG. 26 shows a video / audio reception process operating on the own terminal, and a data transmission process of a video / audio transmission processing apparatus (hereinafter referred to as a counterpart video / audio transmission processing apparatus) to which the camera output and microphone output of the other party are input. The relationship is shown. S171 to S177 indicate video / audio reception processing operating on the terminal, and S178 to S180 indicate data transmission processing of the partner video / audio transmission processing device.

  First, reception processing is started on the own terminal. Modules on the window system necessary for video display and audio output are initialized (S171), and a buffer for receiving data is prepared (S172). A port for reception is opened, and reception is enabled (S173). The other party video / audio transmission processing apparatus is requested to transmit the camera output and microphone output of the other party terminal (S174). Upon receiving this request, the counterpart video / audio transmission processing apparatus, if accepted, executes an initial setting for transmission and requests a communication connection to the reception port on the counterpart terminal (terminal 144a) side (S178). As a result, the terminal (terminal 144a) establishes a communication connection with the partner video / audio transmission processor (S175).

  The counterpart video / audio transmission processing apparatus acquires and encodes video and audio data, stores them in the communication buffer (S179), and transmits the data stored in the communication buffer to the communication partner (terminal 144a) (S180).

  The own terminal (terminal 144a) receives the encoded data (S176), decodes it, displays the video on the window of the video monitor 152a, and outputs the sound from the speaker 154a (S177).

  Thereafter, the partner video / audio transmission processing apparatus repeats S179 and S180, and the terminal 144a also repeats S176 and S177. As a result, video and audio are continuously transferred and reproduced.

  The video output and microphone output of the camera may be branched at an analog signal level and input to both the computers 146a and 146b and the video / audio transmission processing device 140. In this case, it is not necessary to receive own video, but the video capture function for digitizing the camera input is required for the computers 146a and 146b.

  Note that the burden of wiring can be eliminated by wirelessly transmitting the outputs of the cameras 148a and 148b and the microphones 150a and 150b to the video / audio transmission processing device 140.

  With the configuration described above, video / audio information from a plurality of sources within four from a plurality of cameras and microphones can be freely selected and transmitted.

  FIG. 27 shows a schematic block diagram of a modified example of the embodiment shown in FIG. 160 is an 8-input, 4-output matrix switch similar to the matrix switch 112, and 162a, 162b, 162c, 162d captures, encodes, and encodes each of the 4-output video / audio output signals of the matrix switch 160 into the network. The video / audio transmission server 164 to be transmitted is a control device that controls the video / audio transmission servers 162a to 162d and the matrix switch 160 in accordance with an external control signal.

  The audio / video transmission servers 162a to 162d output the outputs of the capture devices 166a to 166d that capture the audio / video output signals of the matrix switch 160, the encoders 168a to 168d and the encoders 168a to 168d that compress and encode the outputs of the capture devices 166a to 166d. It comprises communication buffers 170a to 170d that are temporarily stored for transmission to the network.

  Each of the video / audio transmission servers 162a to 162d takes the output of the matrix switch 160, encodes it, and outputs it to the network, independently of the other video / audio transmission servers 162a to 162d. Therefore, each video / audio information can be transmitted with high quality. In applications that require high frame rate and high resolution signals between multiple points, using multiple encoders as in this embodiment increases cost effectiveness compared to installing capture devices and encoders at each terminal. A plurality of signals can be transmitted simultaneously by distributing the load.

  In the embodiment shown in FIG. 27, to which of a plurality of video / audio transmission servers 162a to 162d a certain transmission processing request is assigned is determined as follows. That is, the transmission requesting side (reception side) refers to the transmission schedulers of the video / audio transmission servers 162a to 162d through the control device 164 of the video / audio transmission processing device on the transmission request receiving side (transmission side). Based on the reference result, the receiving side determines which video / audio transmission server 162a to 162d is requested to perform the transmission processing, and designates the video / audio transmission server 162a to 162d through the control device 164 to instruct the transmission processing.

  Further, when the receiving side requests transmission processing from the transmitting side control device 164 without designation, the transmitting side control device 164 refers to the transmission scheduler of the video / audio transmission servers 162a to 162d and automatically performs the processing according to the load, capability, and the like. Therefore, the processing may be assigned to appropriate video / audio transmission servers 162a to 162d. In this case, there is an effect that the receiving side does not have to bear the burden of referring to the transmission scheduler or specifying the video / audio transmission server.

  In any case, when it is determined which transmission processing request is allocated to which video / audio transmission server 162a to 162d, the allocated transmission processing is added to the transmission scheduler of each video / audio transmission server 162a to 162d. The control device 164 appropriately sets the matrix switch 160 with reference to the transmission scheduler of each of the video / audio transmission servers 162a to 162d. Each of the audio / video transmission servers 162a to 162d takes in each output of the set matrix switch 160, compresses and encodes it, and sends it to the network.

  FIG. 28 shows a schematic block diagram of a video conference system using the video / audio transmission processing apparatus shown in FIG. Reference numeral 180 denotes the video / audio transmission processing apparatus shown in FIG. The same components as those in FIG. 25 are denoted by the same reference numerals. With this configuration, it is possible to distribute the load and simultaneously digitize and compress and encode a plurality of signals without reducing the cost-effectiveness ratio.

  The present invention can be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.) or an apparatus composed of a single device (for example, a copier or a facsimile machine). You may apply.

  In addition, the program code of software for realizing the functions of the embodiment is supplied to an apparatus or a computer in the system connected to the various devices so as to operate the various devices that realize the functions of the embodiment. A device (CPU or MPU) of the apparatus or system that causes the various devices to operate according to a stored program is also included in the technical scope of the present invention. In this case, the program code of the software itself corresponds to realizing the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer, for example, the program code is stored. The storage medium constitutes the present invention.

  As a storage medium for storing the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, and the like can be used.

  Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) or other application software in which the program code is running on the computer, etc. Needless to say, the program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized in cooperation with the embodiment.

  Further, after the supplied program code is stored in a memory provided in a function expansion board of the computer or a function expansion unit connected to the computer, the function code is provided in the function expansion board or function storage unit based on an instruction of the program code. Needless to say, the present invention also includes a case where the CPU or the like performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

It is a schematic block diagram of the first embodiment of the present invention. It is a flowchart of the basic operation | movement of the Example shown in FIG. It is a detailed flowchart of S3 of FIG. It is a detailed flowchart of S5 of FIG. It is a flowchart of the transmission process with respect to a non-occupying process request. It is a flowchart of the initialization process of the transmission process with respect to an exclusive process request. It is a flowchart of the data transmission process with respect to an exclusive process request. It is a schematic block diagram of a video conference system incorporating the embodiment shown in FIG. It is a flowchart of the audio / video transmission process of the other party audio / video transmission processing apparatus, and the audio / video reception process of an own terminal. It is a schematic block diagram of the second embodiment of the present invention. It is a schematic block diagram of a remote monitoring system incorporating the embodiment shown in FIG. It is a schematic block diagram of the example of a change of the system shown in FIG. It is a block diagram showing the 1st Example of this invention using a matrix switch. It is explanatory drawing of the operation mode of the screen division | segmentation unit. 2 is a schematic block diagram of a configuration example of a video encoder 118. FIG. It is a flowchart which shows the operation | movement of the Example of FIG. It is a flowchart of transmission start process S103 of FIG. It is a flowchart of the procedure which adds the process of a non-occupying process request to a transmission scheduler. It is an example of a transmission request sequence. This is a transmission schedule for the transmission request sequence shown in FIG. 17 is a detailed flowchart of a transmission end process S105 in FIG. It is a detailed flowchart of S107 of FIG. 16 in the case of a non-occupying process request. FIG. 17 is a flowchart of the initialization process in S107 of FIG. 16 in the case of an exclusive process request. FIG. 17 is a flowchart of data transmission processing in S107 of FIG. 16 in the case of an exclusive processing request. It is a schematic block diagram of a video conference system using the embodiment shown in FIG. 14 is a flowchart of the video / audio transmission processing of the counterpart video / audio transmission processing device and the video / audio reception processing of the own terminal in the embodiment shown in FIG. 13; It is a schematic block diagram of the 2nd Example of this invention which uses a matrix switch. It is a schematic block diagram of a video conference system using the embodiment shown in FIG.

Explanation of symbols

10: Video / audio transmission processing device 12: Switch 14: Transmission processing device 16: Video capture 18: Video encoder 20: Selector 22: Communication buffer 24: Audio encoder 26: Control circuit 30: Network 40a, 40b: Terminal 42a : Workstation 42b: Personal computer 44a, 44b: Camera 46a, 46b: Microphone 48a, 48b: Video monitor 50a, 50b: Speaker 52: Ethernet 54: Router 110: Video / audio transmission server 112: Matrix switch 114: Screen division Unit 116: Video capture device 118: Video encoder 120: Audio capture device 122: Audio encoder 124: Communication buffer 126: Control circuits 130, 132, 134: Encoder 136: Switch 140: Video Voice transmission processing apparatus 142: communication network 144a, 144b: terminal 146a: Workstation 146b: personal computers 148a, 148b: camera 150a, 150b: microphone 152a, 152 b: video monitor 154a, 154b: speaker 156: Ethernet (registered trademark)
158: Router 160: Matrix switch 162a, 162b, 162c, 162d: Video / audio transmission server 164: Control devices 166a-166d: Capture devices 168a-168d: Encoders 170a-170d: Communication buffers

Claims (18)

A transmission processing device that selects one of a plurality of inputs for at least one of video and audio and transmits the selected video to a network.
Switching means for switching a plurality of analog input signals according to a given instruction;
A / D conversion means for digitizing an analog signal output from the switching means;
Encoding means for compressing and encoding the digital output of the A / D conversion means;
A transmission processing apparatus comprising: output means for outputting data encoded by the encoding means to a network.   The transmission processing according to claim 1, wherein the encoding means includes a plurality of compression encoding means, and the control means selects a compression encoding means to be used based on information exchange with a communication destination and a situation of a communication path. apparatus.   The transmission processing apparatus according to claim 1, wherein the encoding unit includes a video encoding unit and an audio encoding unit.   The transmission processing apparatus according to claim 1, further comprising generating means for generating the instruction. A transmission processing apparatus that selects one of a plurality of inputs for at least one of video and audio and transmits it to a network,
Switching means for switching a plurality of inputs via a network;
Encoding means for compressing and encoding the signal from the switching means;
Output processing means for outputting the encoded output by said means to a network. A switch for selecting an arbitrary plurality of signals from a plurality of analog input signals;
Combining means for compressing and combining a plurality of analog signals selected by the switch on the time axis;
A / D conversion means for digitizing an analog signal output from the synthesis means;
Encoding means for compressing and encoding a digital signal output from the A / D conversion means;
A transmission processing apparatus comprising communication and control means for controlling the switch.   The transmission processing apparatus according to claim 6, wherein the encoding unit includes a plurality of compression encoding units, and the compression encoding unit to be used is selected based on information exchange with a communication destination and a situation of a communication path.   The transmission processing apparatus according to claim 7, wherein information transmitted and received is video information and audio information. A switch for selecting an arbitrary plurality of signals from a plurality of analog input signals;
A plurality of A / D conversion means for digitizing each of a plurality of analog signals selected by the switch;
A plurality of encoding means for compressing and encoding each of the digital signals output from the plurality of A / D conversion means;
A transmission processing apparatus comprising communication and control means for controlling the switch. A transmission processing method for selecting one of a plurality of inputs for at least one of video and audio and transmitting the selected one to a network,
Switch between multiple analog input signals according to given instructions,
Digitized by A / D conversion means,
Compress and encode the digital output,
A transmission processing method, comprising: outputting encoded data to a network.   The transmission processing method according to claim 10, wherein a compression encoding method to be used is selected from a plurality of compression encoding methods based on information exchange with a communication destination and a state of a communication path.   The transmission processing method according to claim 10 or 11, further comprising video encoding means and audio encoding means.   The transmission processing method according to claim 10, further comprising generating means for generating the instruction. A transmission processing method for selecting one of a plurality of inputs for at least one of video and audio and transmitting it to a network,
Switching between multiple inputs over the network, compression encoding,
A transmission processing method characterized by outputting an encoded output to a network. Select any multiple signals from multiple analog input signals,
Compress and synthesize selected analog signals on the time axis,
Digitize the synthesized analog signal,
A transmission processing method characterized by compression-coding a digitized composite signal.   The transmission processing method according to claim 16, wherein a compression encoding method to be used is selected from a plurality of compression encoding methods based on information exchange with a communication destination and a state of a communication path.   The transmission processing method according to claim 16, wherein the information transmitted and received is video information and audio information. Select any multiple signals from multiple analog input signals,
Digitize each of the selected analog signals,
A transmission processing method characterized by compression-coding each digital signal.

Images