JP2006033396A - Transmission device, reception device, communication system, transmission method, reception method, transmission program, reception program, and server device - Google Patents

Abstract

【Task】
To provide a transmission device, a reception device, a communication system, a transmission method, a reception method, these programs, and a server device that can communicate data such as moving images and voices in an optimal environment regardless of network conditions.
[Solution]
The transmission apparatus 100 according to the present invention selects an optimal codec among the first, second, and third codecs 39a, 39b, and 39c in accordance with the network conditions that change depending on the time zone and the reception apparatus 200 that is the connection partner. Since the selection is made, the receiving apparatus 200 can always receive data with the optimum image quality and sound quality. Therefore, even if packet burst loss occurs, smooth communication can be performed.
[Selection] Figure 9

Description

  The present invention relates to a transmission device, a reception device, a communication system, a transmission method, a reception method, a transmission program, a reception program, and a server device that transmit or receive data such as moving images and audio.

  Conventionally, when transmitting and receiving data such as moving images and sounds, data encoded according to a codec of a transmission device is packetized, and the packet is transmitted to a reception device via a network such as the Internet. The receiving apparatus decodes and reproduces the received packet data with a decoder according to the codec.

  In this way, when packets of data such as moving images and voices are exchanged on the network, packet loss, transmission delay, and the like occur depending on network congestion conditions and router performance. When the packet loss and the transmission delay increase in this way, for example, in a video conference system, the real-time property is impaired, and there is a problem that the image quality and the sound quality are deteriorated or, for example, a part of the moving image data is lost. On a network where the bandwidth (bit rate) etc. set by the user in advance in the transmitting device and the receiving device differ depending on the time, the receiving side is not necessarily able to receive high-quality and high-quality sound data only by the initial setting at the time of connection. Not exclusively.

In order to solve such a problem, conventionally, there is a technique in which the bit rate is dynamically changed or the receiving side retransmits the lost packet (for example, refer to Patent Documents 1 and 2).
JP 2003-244695 A (paragraphs [0049] to [0054]) Japanese Patent Laying-Open No. 2003-169040 (paragraph [0060], FIG. 4 and FIG. 11)

  However, if the bit rate is changed, the bit rate is lowered with respect to the codec set in the transmission device and the reception device, so communication that satisfies the frame rate cannot be performed, and communication becomes awkward. In addition, since the network status varies depending on the time zone and the device on the connection partner side, it is difficult for a general user to determine whether or not the set bit rate or the like is optimal.

  In particular, in MCU (Multi-point Control Unit) connection used in a video conference system, communication is performed after sharing the capability of each terminal device in the negotiation at the time of connection between the MCU server and each terminal device. Therefore, if one terminal device supports a codec with high image quality and high sound quality, but the other terminal device has only a codec with low image quality and low sound quality, that one terminal device is the expected one. I can't show my ability. In addition, even when the bit rate is changed, a terminal with a light network load is received together with the changed data, which affects the image quality. In other words, in the conventional MCU system, it is not possible to deliver high-function moving images, sounds, and the like unless the terminal devices having high functions are connected to each other.

  In view of the circumstances as described above, an object of the present invention is to provide a transmission device, a reception device, a communication system, a transmission method, and a communication device that can perform data communication such as video and audio in an always optimal environment regardless of network conditions. It is in providing a receiving method, these programs, and a server apparatus.

  In order to achieve the above object, a transmitting apparatus according to the present invention includes a first encoder that encodes data using a first codec, and a second codec that is different from the first codec. A second encoder for encoding, a transmission means capable of transmitting data encoded by the first and second encoders to a receiving apparatus connected via a network, and a monitoring means for monitoring the status of the network And selection means for selecting which of the first and second encoders the data is to be encoded and transmitted by the transmission means according to at least the status of the monitored network. To do.

  In the present invention, the optimal codec is selected from the first codec and the second codec in accordance with the network conditions that change depending on the time zone and the receiving device that is the connection partner. Data can be received with sound quality. Here, the “second codec different from the first codec” means that even if the standard and format are the same, if the algorithm or program is different, the codec is different. The network status is information such as transmission delay time and data loss rate on the network, for example. The same applies hereinafter.

  According to an aspect of the present invention, the monitoring unit includes a unit that acquires a loss rate of data transmitted on the network, and the selection unit is configured to select the first rate according to the acquired loss rate. The encoder selects which of the encoders encodes the data and transmits the data by the transmission means. When the transmission device acquires the loss rate, it is possible to grasp a more accurate network state and transmit data at an optimum bit rate.

  According to an aspect of the present invention, the monitoring means includes means for detecting whether a loss of data transmitted on the network is a random loss or a burst loss, and the selection means includes the detection In accordance with the result, it is selected which of the first and second encoders encodes the data and transmits the data by the transmission means. Optimal communication is possible by using the first and second codecs in accordance with the random loss and the burst loss. Specifically, for example, it becomes possible for the transmission device to measure the transmission delay time of network data, or to detect whether it is a random loss or a burst loss. Alternatively, the transmission device can detect whether it is a random loss or a burst loss by calculating the packet loss rate and sending the loss rate information to the transmission device. In the case of burst loss, the loss rate within a certain time period or within a certain unit time is considerably higher than in the case of random loss, so that both can be detected.

  The transmission apparatus according to an aspect of the present invention further includes means for transmitting data lost due to the random loss to the reception apparatus when the random loss is detected by the monitoring means. Specifically, when a signal requesting retransmission of data lost due to random loss is received from the receiving device, the monitoring unit can detect that the random loss has occurred. When a random loss occurs, the network is not so heavily loaded, so even if data is retransmitted, the network is not congested.

  The transmission device according to an aspect of the present invention further includes means for acquiring status information including at least one of the capabilities and settings of the reception device from the reception device via the network, and the selection The means has means for selecting which of the first and second encoders encodes the data and transmits the data by the transmission means in accordance with the acquired status information. Thereby, optimal communication is possible according to the status information of the receiving device.

  The capability in the status information is information such as the processor clock frequency, the current processor usage rate, the total capacity of the memory (including the external memory), the current memory usage, and the like. The setting in the status information is user setting information for the receiving device when the transmitting device and the receiving device are first connected (initial state). After that, the information is automatically set according to the network status. These pieces of setting information are setting information such as codec and bit rate. In the initial state, the setting information includes user settings such as which one of image quality, sound quality, network bandwidth, etc. to prioritize. The same applies hereinafter. For example, in order to be able to reproduce without generating packet loss even if the image quality is low, the network bandwidth may be prioritized. That is, a low bit rate is set in order to provide a sufficient bandwidth.

  The transmission apparatus according to an aspect of the present invention further includes means for monitoring a load status of the transmission apparatus, and the selection means is configured to change the first and second in accordance with the monitored load status. The encoder has means for selecting which of the encoders encodes the data and transmits the data by the transmitting means. Thereby, for example, when the load on the transmission apparatus is large, the codec for low bit rate is used, so that the load on the processor can be reduced. Whether the load is large or not can be determined, for example, based on whether the processor, memory usage rate, etc. of the transmission device exceed a threshold value.

  A transmitting apparatus according to another aspect of the present invention encodes a first encoder that encodes data using a first codec and a second codec that is different from the first codec. Two encoders, transmitting means capable of transmitting data encoded by the first and second encoders to a receiving apparatus connected via a network, and at least one of the capabilities and settings of the receiving apparatus Means for acquiring status information including information from the receiving device via the network, and depending on the acquired status information, the encoder is encoded by any one of the first and second encoders; Selecting means for selecting whether to transmit data by the transmitting means.

  In the present invention, since the optimal codec is selected from the first codec and the second codec in accordance with the status information of the receiving apparatus, the receiving apparatus can always receive data with the optimal image quality and sound quality.

  A receiving apparatus according to the present invention includes a first encoder that encodes data using a first codec, and a second encoder that encodes the data using a second codec different from the first codec. , Transmission means capable of transmitting the first and second data encoded by the first and second encoders, respectively, over the network, monitoring means for monitoring the status of the network, and at least the network to be monitored Depending on the situation, the network may be transmitted from a transmission device having a first selection unit that selects which of the first and second encoders encodes the data and transmits the data by the transmission unit. Means for receiving either one of the first data and the second data transmitted via the first data; and the first code. A first decoder that decodes the first data using the first code, a second decoder that decodes the second data using the second codec, and the first and second data Of these, a second selection means for selecting which one of the first and second decoders to decode according to received data is provided.

  In the present invention, the optimal codec of the first codec and the second codec is selected and encoded by the transmitting device according to the network conditions, and the receiving device selects and decodes the decoder corresponding to this codec. To do. As a result, the receiving apparatus can always reproduce data with the optimum image quality and sound quality.

  A communication system according to the present invention includes a first encoder that encodes data using a first codec, and a second encoder that encodes data using a second codec different from the first codec. , Transmission means capable of transmitting data encoded by the first and second encoders over a network, monitoring means for monitoring the status of the network, and at least according to the status of the monitored network, A transmission device having first selection means for selecting which of the first and second encoders encodes the data and transmits the data by the transmission means; and the transmission device that is transmitted via the network Means for receiving one of the first and second data, and the first codec A first decoder that decodes the first data; a second decoder that decodes the second data using the second codec; and data received from the first and second data And a receiving device having second selecting means for selecting which one of the first and second decoders to decode.

  A transmission method according to the present invention is a method in which a transmission device connected to a reception device via a network transmits data, and the step of monitoring the status of the network and at least according to the status of the monitored network The encoder encodes data using a first encoder that encodes data using a first codec, and a second encoder that encodes data using a second codec different from the first codec. And a step of transmitting the data encoded by the selected encoder to the receiving device.

  A receiving method according to the present invention includes a first encoder that encodes data using a first codec, and a second encoder that encodes the data using a second codec different from the first codec. , Transmission means capable of transmitting the first and second data encoded by the first and second encoders, respectively, over the network, monitoring means for monitoring the status of the network, and at least the network to be monitored Depending on the situation, transmission is performed via the network from a transmission device having selection means for selecting which of the first and second encoders encodes the data and causes the transmission means to transmit the data. Receiving one of the first data and the second data, and the first and second data Depending on the data received in the data, comprising the steps of selecting whether to decode any of the decoder of the first and the second decoder.

  A transmission program according to the present invention is a program in which a transmission device connected to a reception device via a network transmits data, wherein the transmission device monitors the status of the network, and at least the monitored A first encoder that encodes data using a first codec and a second encoder that encodes the data using a second codec different from the first codec according to network conditions A step of selecting which encoder to encode and a step of transmitting the data encoded by the selected encoder to the receiving device are executed.

  A reception program according to the present invention encodes, in a receiving device, a first encoder that encodes data using a first codec, and a second codec that is different from the first codec. Two encoders, transmission means capable of transmitting the first and second data encoded by the first and second encoders over a network, monitoring means for monitoring the status of the network, and at least the monitoring A transmission unit having a selection unit that selects which of the first and second encoders encodes the data and transmits the encoded data by the transmission unit according to a situation of the network to be transmitted; Receiving either one of the first and second data transmitted via The first and in accordance with the data received in the second data, and a step of selecting whether to decode any of the decoder of the first and second decoder.

  A server apparatus according to the present invention includes a first encoder that encodes data using a first codec, and a second encoder that encodes the data using a second codec different from the first codec. Status information including information on at least one of the capabilities and settings of each of a plurality of receiving apparatuses connected via a network based on the status information, and means for acquiring status information from each receiving apparatus via the network , When it is determined that the first and second receiving devices respectively have first and second decoders that decode using the first and second codecs, respectively, Means for controlling to transmit data to the first and second receiving devices, respectively.

  In the present invention, the first and second receiving apparatuses respectively have first and second decoders that decode using the first and second codecs based on the acquired status information for each receiving apparatus. Since the first and second data are controlled to be transmitted to the first and second receiving devices, respectively, each receiving device can communicate in an optimum environment. The present invention is particularly preferably applied to an MCU.

  The server device according to an aspect of the present invention further includes means for determining whether both the first and second encoders are usable or only the first encoder is usable, and the control means includes , When it is determined that only the first encoder is usable, and when it is determined that the second receiving device has the first decoder based on the status information, the first data is Control is performed to transmit to the first and second receiving apparatuses. For example, the load on the server device can be reduced by assigning the codec of each receiving device according to the load status of the server device or according to the capability of the server device.

  A server apparatus according to an aspect of the present invention includes a network monitoring unit that monitors a status of the network, and at least the data in the first and second encoders according to the status of the monitored network. And a means for selecting which encoder to encode and to transmit by the transmission means. As a result, each receiving apparatus can always receive data with optimum image quality and sound quality regardless of the network conditions.

  As described above, according to the present invention, data such as moving images and voices can be communicated in an optimum environment at all times regardless of the network conditions.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, a network system as a premise in one embodiment of the present invention will be described. FIG. 1 shows an example of a network system according to the present invention, in which a plurality of information processing apparatuses 1, 2, 3, 4 are connected via a network 9. The information processing devices 1, 2, 3, 4 are, for example, various AV (Audio and Visual) devices, portable devices, and the like.

  As for the information processing apparatus 1, the information processing apparatus 1 includes an information processing controller 11 as a computer function unit. The information processing controller 11 includes a main processor 21-1, sub processors 23-1, 23-2, and 23-3, a DMAC (direct memory access controller) 25-1, and a DC (disk controller) 27-1.

  The main processor 21-1 performs schedule management of program execution (data processing) by the sub processors 23-1, 23-2, and 23-3 and general management of the information processing controller 11 (information processing apparatus 1). . However, a program other than the management program may be operated in the main processor 21-1. In that case, the main processor 21-1 also functions as a sub processor. The main processor 21-1 has an LS (local storage) 22-1.

  There may be one sub-processor, but preferably there are a plurality of sub-processors. This example is a plurality of cases. Each of the sub processors 23-1, 23-2, and 23-3 executes a program in parallel and independently under the control of the main processor 21-1, and processes data. Further, in some cases, the program in the main processor 21-1 can be configured to operate in cooperation with the programs in the sub-processors 23-1, 23-2, and 23-3. A function program described later is also a program that operates in the main processor 21-1. The sub processors 23-1, 23-2, and 23-3 also have LS (local storage) 24-1, 24-2, and 24-3.

  The DMAC 25-1 accesses a program and data stored in a main memory 26-1 including a DRAM (dynamic RAM) connected to the information processing controller 11, and the DC 27-1 is used for the information processing controller 11. Are accessed to the external recording units 28-1 and 28-2.

  The external recording units 28-1 and 28-2 may be fixed disks (hard disks) or removable disks, and are optical disks such as MO, CD ± RW, DVD ± RW, memory disks, SRAM (static RAM), ROM, and the like. Various types can be used. Therefore, although DC27-1 is called a disk controller, it is an external recording unit controller. As in the example of FIG. 1, the information processing controller 11 can be configured so that a plurality of external recording units 28 can be connected to the information processing controller 11.

  The main processor 21-1, the sub processors 23-1, 23-2, and 23-3, the DMAC 25-1, and the DC 27-1 are connected by a bus 29-1.

  An identifier that can uniquely identify the information processing apparatus 1 including the information processing controller 11 throughout the entire network is assigned to the information processing controller 11 as an information processing apparatus ID.

  Similarly, an identifier that can identify each of the main processor 21-1 and each of the sub processors 23-1, 23-2, and 23-3 is assigned as a main processor ID and a sub processor ID.

  The information processing controller 11 is preferably configured as a one-chip IC (integrated circuit). Other information processing apparatuses 2, 3, and 4 are configured in the same manner as described above. Here, the units having the same parent number in FIG. 1 perform the same function even if the branch numbers are different unless otherwise noted. Further, in the following description, when the branch number is omitted, it is assumed that the difference in the branch number does not occur.

  As described above, each sub-processor 23 in one information processing controller independently executes a program and processes data, but different sub-processors simultaneously read or write to the same area in the main memory 26. If done, data inconsistencies can occur. Therefore, access from the sub processor 23 to the main memory 26 is performed according to the following procedure.

  As shown in FIG. 2A, the main memory 26 is composed of memory locations that can specify a plurality of addresses. Each memory location is allocated an additional segment for storing information indicating the state of the data. The additional segment includes an F / E bit, a sub processor ID, and an LS address (local storage address). Each memory location is also assigned an access key to be described later. The F / E bit is defined as follows.

  The F / E bit = 0 indicates that the data being processed being read by the sub-processor 23 or invalid data that is not the latest data because it is empty and cannot be read. The F / E bit = 0 indicates that data can be written to the memory location, and is set to 1 after writing.

  The F / E bit = 1 indicates that the data at the memory location has not been read by the sub-processor 23 and is the latest unprocessed data. The data in the memory location can be read and set to 0 after being read by the sub-processor 23. Further, the F / E bit = 1 indicates that the memory location cannot write data.

  Furthermore, it is possible to set a read reservation for the memory location in the state where the F / E bit = 0 (reading impossible / writing possible). When a read reservation is made for a memory location with the F / E bit = 0, the sub processor 23 adds the sub processor ID and LS of the sub processor 23 as read reservation information to an additional segment of the memory location where the read reservation is made. Write the address.

  Thereafter, when data is written to the memory location reserved for reading by the sub-processor 23 on the data writing side and the F / E bit is set to 1 (readable / not writable), it is preliminarily stored in the additional segment as read reservation information. Read to the written sub-processor ID and LS address.

  When it is necessary to process data in multiple stages by a plurality of sub-processors, the sub-processor 23 that performs the process in the previous stage controls the read / write of the data in each memory location in this way. Immediately after the data is written at a predetermined address on the main memory 26, another sub-processor 23 that performs the subsequent processing can read the data after the preprocessing.

  As shown in FIG. 2B, the LS 24 in each sub-processor 23 is also configured by memory locations that can specify a plurality of addresses. An additional segment is similarly allocated for each memory location. The additional segment includes a busy bit.

  When the sub-processor 23 reads the data in the main memory 26 to the memory location of its own LS 24, it reserves by setting the corresponding busy bit to 1. No other data can be stored in the memory location where the busy bit is 1. After reading to the memory location of the LS 24, the busy bit becomes 0 and can be used for any purpose.

  As shown in FIG. 2A, the main memory 26 connected to each information processing controller further includes a plurality of sandboxes. The sandbox defines an area in the main memory 26, and each sandbox is assigned to each sub processor 23 and can be used exclusively by the sub processor. That is, each sub-processor 23 can use a sandbox assigned to itself, but cannot access data beyond this area. The main memory 26 is composed of a plurality of memory locations, and the sandbox is a set of these memory locations.

  Furthermore, in order to realize exclusive control of the main memory 26, a key management table as shown in FIG. 2C is used. The key management table is stored in a relatively high-speed memory such as SRAM in the information processing controller, and is associated with the DMAC 25. Each entry in the key management table includes a sub processor ID, a sub processor key, and a key mask.

  The process when the sub processor 23 uses the main memory 26 is as follows. First, the sub processor 23 outputs a read or write command to the DMAC 25. This command includes its own sub-processor ID and the address of the main memory 26 that is the use request destination.

  Before executing this command, the DMAC 25 refers to the key management table and checks the sub processor key of the sub processor of the use request source. Next, the DMAC 25 compares the checked sub-processor key of the use request source with the access key allocated to the memory location shown in FIG. Execute the above command only when two keys match.

  In the key mask on the key management table shown in FIG. 2C, when the arbitrary bit becomes 1, the corresponding bit of the sub-processor key associated with the key mask may become 0 or 1. it can. For example, assume that the sub-processor key is 1010. Normally, this sub-processor key only allows access to a sandbox with 1010 access keys. However, if the key mask associated with this sub-processor key is set to 0001, the match determination between the sub-processor key and the access key is masked only for the digit whose key mask bit is set to 1. This sub-processor key 1010 enables access to a sandbox having an access key whose access key is either 1010 or 1011.

  As described above, the sandbox exclusivity of the main memory 26 is realized. That is, when it is necessary to process data in multiple stages by a plurality of sub-processors in one information processing controller, by configuring as described above, the sub-processor that performs the process in the previous stage and the process in the subsequent stage are processed. Only the sub processor that performs the access can access a predetermined address in the main memory 26, and data can be protected.

  For example, it can be used as follows. First, immediately after the information processing apparatus is activated, the values of the key masks are all zero. It is assumed that a program in the main processor is executed and operates in cooperation with a program in the sub processor. When the processing result data output by the first sub-processor is temporarily stored in the main memory and desired to be input to the second sub-processor, the corresponding main memory area must naturally be accessible from either sub-processor. is there. In such a case, the program in the main processor appropriately changes the value of the key mask and provides a main memory area that can be accessed from a plurality of sub processors, thereby enabling multi-stage processing by the sub processors. .

More specifically, it is a multi-step process in the order of data from another information processing apparatus → processing by the first sub processor → first main memory area → processing by the second sub processor → second main memory area. When processing is done,
Sub-processor key of the first sub-processor: 0100
First main memory area access key: 0100,
Sub-processor key of the second sub-processor: 0101,
Access key for second main memory area: 0101
In such a setting, the second sub-processor cannot access the first main memory area. Therefore, by setting the key mask of the second sub processor to 0001, it is possible to allow the second sub processor to access the first main memory area.

  In the network system of FIG. 1, software cells are transmitted between the information processing apparatuses 1, 2, 3, and 4 for distributed processing between the information processing apparatuses 1, 2, 3, and 4. That is, the main processor 21 included in the information processing controller in a certain information processing apparatus generates a software cell including a command, a program, and data, and transmits it to another information processing apparatus via the network 9 to perform processing. Can be dispersed.

  FIG. 3 shows an example of the configuration of the software cell. The software cell in this example is composed of a transmission source ID, a transmission destination ID, a response destination ID, a cell interface, a DMA command, a program, and data as a whole.

  The transmission source ID includes the network address of the information processing apparatus that is the transmission source of the software cell, the information processing apparatus ID of the information processing apparatus, and the main processor 21 and each sub processor included in the information processing controller in the information processing apparatus. 23 identifiers (main processor ID and sub-processor ID) are included.

  The transmission destination ID and the response destination ID include the same information about the information processing apparatus that is the transmission destination of the software cell and the information processing apparatus that is the response destination of the execution result of the software cell, respectively.

  The cell interface is information necessary for using the software cell, and includes a global ID, necessary sub-processor information, a sandbox size, and a previous software cell ID.

  The global ID can uniquely identify the software cell throughout the network, and is created based on the transmission source ID and the date and time (date and time) of creation or transmission of the software cell.

  The necessary sub-processor information sets the number of sub-processors necessary for executing the software cell. The sandbox size sets the amount of memory in the main memory 26 and the LS 24 of the sub processor 23 necessary for executing the software cell. The previous software cell ID is an identifier of the previous software cell in a group of software cells that request sequential execution of streaming data or the like.

  The execution section of the software cell is composed of DMA commands, programs, and data. The DMA command includes a series of DMA commands necessary for starting the program, and the program includes a sub processor program executed by the sub processor 23. The data here is data processed by a program including the sub processor program.

  Further, the DMA command includes a load command, a kick command, a function program execution command, a status request command, and a status return command.

  The load command is a command for loading information in the main memory 26 into the LS 24 in the sub processor 23, and includes a main memory address, a sub processor ID, and an LS address in addition to the load command itself. The main memory address indicates an address of a predetermined area in the main memory 26 from which information is loaded. The sub processor ID and the LS address indicate the identifier of the sub processor 23 to which the information is loaded and the address of the LS 24.

  The kick command is a command for starting execution of a program, and includes a sub processor ID and a program counter in addition to the kick command itself. The sub processor ID identifies the sub processor 23 to be kicked, and the program counter gives an address for the program execution program counter.

  As will be described later, the function program execution command is a command for requesting execution of a function program from another information processing apparatus to another information processing apparatus. The information processing controller in the information processing apparatus that has received the function program execution command identifies a function program to be activated by a function program ID described later.

  The status request command is a command for requesting transmission of device information related to the current operation state (situation) of the information processing device indicated by the transmission destination ID to the information processing device indicated by the response destination ID. Although the function program will be described later, it is a program categorized into the function program in the software configuration diagram stored in the main memory 26 of the information processing controller shown in FIG. The function program is loaded into the main memory 26 and executed by the main processor 21.

  The status reply command is a command in which the information processing apparatus that has received the status request command responds to the information processing apparatus indicated by the response destination ID included in the status request command with its own apparatus information. The status reply command stores device information in the data area of the execution section.

  FIG. 4 shows the structure of the data area of the software cell when the DMA command is a status return command.

  The information processing device ID is an identifier for identifying the information processing device including the information processing controller, and indicates the ID of the information processing device that transmits the status reply command. The information processing device ID is included in the information processing controller in the information processing device by the main processor 21 included in the information processing controller in the information processing device when the power is turned on. It is generated based on the number of sub processors 23 to be processed.

  The information processing device type ID includes a value representing the characteristics of the information processing device. The characteristics of the information processing apparatus are, for example, a hard disk recorder, a PDA (Personal Digital Assistants), a portable CD (Compact Disc) player, and the like. The information processing device type ID may represent a function of the information processing device such as video / audio recording or video / audio reproduction. It is assumed that values representing the characteristics and functions of the information processing apparatus are determined in advance, and it is possible to grasp the characteristics and functions of the information processing apparatus by reading the information processing apparatus type ID.

  The MS (master / slave) status indicates whether the information processing apparatus is operating as a master apparatus or a slave apparatus, as will be described later. When this is set to 0, it operates as a master apparatus. If it is set to 1, it indicates that it is operating as a slave device.

  The main processor operating frequency represents the operating frequency of the main processor 21 in the information processing controller. The main processor usage rate represents the usage rate in the main processor 21 for all programs currently running on the main processor 21. The main processor usage rate is a value representing the ratio of the processing capacity in use to the total processing capacity of the target main processor. For example, the main processor usage rate is calculated by using MIPS as a unit for evaluating the processor processing capacity, or per unit time. Calculated based on processor usage time. The same applies to the sub-processor usage rate described later.

  The number of sub-processors represents the number of sub-processors 23 included in the information processing controller. The sub processor ID is an identifier for identifying each sub processor 23 in the information processing controller.

  The sub processor status represents the state of each sub processor 23, and there are states such as “unused”, “reserved”, and “busy”. “unused” indicates that the sub-processor is not currently used and is not reserved for use. “reserved” indicates a reserved state that is not currently used. Busy indicates that it is currently in use.

  The sub-processor usage rate represents the usage rate of the sub-processor for a program that is currently being executed by the sub-processor or that is reserved for execution by the sub-processor. That is, the sub processor usage rate indicates the current usage rate when the sub processor status is busy, and indicates the estimated usage rate that is to be used later when the sub processor status is reserved.

  One set of sub processor ID, sub processor status, and sub processor usage rate is set for one sub processor 23, and the number of sets corresponding to the sub processor 23 in one information processing controller is set.

  The total main memory capacity and the main memory usage represent the total capacity and the currently used capacity of the main memory 26 connected to the information processing controller, respectively.

  The number of external recording units represents the number of external recording units 28 connected to the information processing controller. The external recording unit ID is information that uniquely identifies the external recording unit 28 connected to the information processing controller. The external recording unit type ID represents the type of the external recording unit (for example, hard disk, CD ± RW, DVD ± RW, memory disk, SRAM, ROM, etc.).

  The external recording unit total capacity and the external recording unit usage amount represent the total capacity and the currently used capacity of the external recording unit 28 identified by the external recording unit ID, respectively.

  The external recording unit ID, the external recording unit type ID, the external recording unit total capacity, and the external recording unit usage amount are set for one external recording unit 28 and connected to the information processing controller. The number of sets corresponding to the number of external recording units 28 is set. That is, when a plurality of external recording units are connected to one information processing controller, a different external recording unit ID is assigned to each external recording unit, the external recording unit type ID, the external recording unit total capacity, and the external recording unit Department usage is also managed separately.

  The main processor 21 included in the information processing controller in a certain information processing device generates a software cell having the above configuration and transmits it to the other information processing device and the information processing controller in the device via the network 9. . The transmission source information processing device, the transmission destination information processing device, the response destination information processing device, and the information processing controller in each device are identified by the transmission source ID, the transmission destination ID, and the response destination ID, respectively. .

  The main processor 21 included in the information processing controller in the information processing apparatus that has received the software cell stores the software cell in the main memory 26. Furthermore, the main processor 21 of the transmission destination reads the software cell and processes the DMA command included therein. Specifically, the transmission destination main processor 21 first executes a load command. As a result, information is loaded from the main memory address instructed by the load command into a predetermined area of the LS 24 in the sub processor identified by the sub processor ID and LS address included in the load command. The information loaded here is a sub-processor program or data included in the received software cell, or other designated data.

  Next, the main processor 21 outputs the kick command together with the program counter included in the kick command to the sub processor indicated by the sub processor ID included therein. The instructed sub processor executes the sub processor program according to the kick command and the program counter. After the execution result is stored in the main memory 26, the main processor 21 is notified that the execution has been completed.

  Note that the processor that executes the software cell in the information processing controller in the information processing apparatus of the transmission destination is not limited to the sub-processor 23, but the main processor 21 executes a program for main memory such as a function program included in the software cell. It can also be specified to execute.

  In this case, the transmission source information processing apparatus includes a main memory program and data processed by the main memory program instead of the sub processor program, and the DMA command is sent to the transmission destination information processing apparatus. A software cell as a load command is transmitted, and the main memory 26 stores the main memory program and data processed thereby. Next, the transmission source information processing apparatus identifies the main processor ID, the main memory address, and the main memory program for the information processing controller in the transmission destination information processing apparatus for the transmission destination information processing apparatus. A software cell that includes an identifier such as a function program ID (to be described later) and a program counter and whose DMA command is a kick command or a function program execution command is transmitted to cause the main processor 21 to execute the main memory program.

  As described above, in the network system of the present invention, the transmission source information processing apparatus transmits the sub processor program or the main memory program to the transmission destination information processing apparatus by the software cell, and transmits the sub processor program to the transmission destination. It is possible to load the sub processor 23 included in the information processing controller in the information processing apparatus and cause the information processing apparatus of the transmission destination to execute the sub processor program or the main memory program.

  When the program included in the received software cell is a sub processor program, the information processing controller in the transmission destination information processing apparatus loads the sub processor program to the designated sub processor. Then, the sub processor program or the main memory program included in the software cell is executed. Therefore, even if the user does not operate the transmission destination information processing apparatus, the sub processor program or the main memory program can be automatically executed by the information processing controller in the transmission destination information processing apparatus.

  In this way, when the information processing controller in its own device does not have a main memory program such as a sub processor program or a function program, the information processing device can receive information from other information processing devices connected to the network. Can be obtained. Furthermore, data is transferred between the sub-processors by the DMA method, and the above-described sandbox is used, so that even when it is necessary to process data in multiple stages within one information processing controller, high speed and high security are achieved. The process can be executed.

  As a result of distributed processing using software cells, a plurality of information processing apparatuses 1, 2, 3, 4 connected to the network 9 as shown in the upper part of FIG. It operates as a single information processing device 7. However, for this purpose, the following processing needs to be executed by the following configuration.

  FIG. 6 shows the configuration of software stored in the main memory 26 of each information processing controller. These software (programs) are recorded in the external recording unit 28 connected to the information processing controller before the information processing apparatus is turned on. Each program is categorized into a control program, a function program, and a device driver according to functions or features.

  The control program is the same for each information processing controller, and is executed by the main processor 21 of each information processing controller, and includes an MS (master / slave) manager and a capacity exchange program described later.

  The function program is executed by the main processor 21, and a function program corresponding to the information processing apparatus is provided for each information processing controller such as recording, reproduction, and material search.

  The device driver is for input / output (transmission / reception) of the information processing controller (information processing apparatus), such as broadcast reception, monitor output, bit stream input / output, network input / output, etc. Provided.

  When the information processing apparatus is physically connected to the network 9 and the main power is turned on, and the information processing apparatus is electrically and functionally connected to the network 9, the information processing apparatus The main processor 21 of the information processing controller loads each program belonging to the control program and each program belonging to the device driver into the main memory 26.

  As a loading procedure, the main processor 21 first reads a program from the external recording unit 28 by causing the DC 27 to execute a read command, and then causes the DMAC 25 to execute a write command to load the program into the main memory 26. Write to.

  As for each program belonging to the function program, it may be configured to load only the necessary program when necessary, or like the programs belonging to other categories, each program is loaded immediately after the main power is turned on. You may comprise as follows.

  Here, each program belonging to the function program does not need to be recorded in the external recording unit 28 of all information processing apparatuses connected to the network, and is recorded in the external recording unit 28 of any one information processing apparatus. If so, it can be loaded from another information processing apparatus by the above-described method. As a result, the function program is executed as one virtual information processing apparatus 7 as shown in the lower part of FIG. Can do.

  Here, as described above, the function program processed by the main processor 21 may operate in cooperation with the sub processor program processed by the sub processor 23. Therefore, when there is a sub processor program that operates in cooperation with the target function program when the main processor 21 reads the function program from the external recording unit 28 and writes it to the main memory 26, the sub processor program also includes the same main program. It is assumed that data is written in the memory 26. In this case, there may be one or more sub-processor programs that operate in cooperation with each other. If there are a plurality of sub-processor programs that operate in cooperation, all the sub-processor programs are written in the main memory 26. The sub processor program written in the main memory 26 is then written in the LS 24 in the sub processor 23 and operates in cooperation with the function program processed by the main processor 21.

  As shown in the software cell of FIG. 3, an identifier that can uniquely identify a program for each program is assigned to the function program as a function program ID. The function program ID is determined from the creation date and time, the information processing apparatus ID, and the like at the stage of creating the function program.

  A sub processor program ID is also assigned to the sub processor program, whereby the sub processor program can be uniquely identified. The assigned sub-processor program ID is an identifier related to the function program ID of the function program that is the partner of the cooperative operation, for example, the function program ID as a parent number and a branch number added at the end. There may be an identifier that is not related to the function program ID of the function program that is the partner of the cooperative operation. In any case, when the function program and the sub processor program operate in cooperation, it is necessary to store the program ID which is the identifier of the other party in the own program. Even when the function program operates in cooperation with a plurality of sub processor programs, the function program stores the sub processor program IDs of all the sub processor programs.

  The main processor 21 secures an area for storing device information (information regarding the operation state) of the information processing device on which the main processor 21 operates in the main memory 26, and records the information as a device information table of the own device. The device information here is each piece of information below the information processing device ID shown in FIG.

  In the network system described above, when the main power supply to a certain information processing apparatus is turned on, the main processor 21 of the information processing controller of the information processing apparatus loads a master / slave manager (hereinafter referred to as MS manager) into the main memory 26 and executes it. To do.

  When the MS manager detects that the information processing apparatus on which it operates is connected to the network 9, it confirms the existence of another information processing apparatus connected to the same network 9. The “connection” or “existence” here means that the information processing apparatus is not only physically connected to the network 9 but also electrically and functionally connected to the network 9 as described above. Indicates. In addition, an information processing apparatus in which the device operates is referred to as a self device, and another information processing device is referred to as another device. The apparatus also indicates the information processing apparatus.

  A method in which the MS manager confirms the existence of another information processing apparatus connected to the same network 9 will be described below.

  The MS manager generates a software cell in which the DMA command is a status request command, the transmission source ID and the response destination ID are the information processing apparatus, and the transmission destination ID is not specified, and the network manager is connected to the information processing apparatus. To set a timer for network connection confirmation. The timeout time of the timer is, for example, 10 minutes.

  When another information processing apparatus is connected to the network system, the other apparatus receives the software cell of the status request command, and sends it to the information processing apparatus that has issued the status request command specified by the response destination ID. On the other hand, the DMA command is a status return command, and a software cell including device information of itself (other device) is transmitted as data. The software cell of the status reply command includes at least information for identifying the other device (information processing device ID, information on the main processor, information on the sub processor, etc.) and the MS status of the other device.

  The MS manager of the information processing apparatus that has issued the status request command monitors the reception of the software cell of the status reply command transmitted from another apparatus on the network until the timer for network connection confirmation times out. As a result, when the status reply command indicating the MS status = 0 (master device) is received, the MS status in the device information table of the own device is set to 1. Thus, the device becomes a slave device.

  On the other hand, if no status reply command is received before the network connection confirmation timer times out, or if no status reply command indicating MS status = 0 (master device) is received, The MS status in the device information table of the own device is set to 0. This makes the device a master device.

  That is, if no information processing apparatus is connected to the network 9 in a state where none of the apparatuses is connected to the network 9 or a master apparatus does not exist on the network 9, the apparatus automatically becomes the master apparatus. Set as On the other hand, when a new information processing apparatus is connected to the network 9 in a state where a master apparatus already exists on the network 9, the apparatus is automatically set as a slave apparatus.

  For both the master device and the slave device, the MS manager periodically monitors the status of the other device by sending a status request command to the other device on the network 9 and inquiring status information. As a result, the main power supply of the information processing apparatus connected to the network 9 is cut off or the information processing apparatus is disconnected from the network 9, so that the status from a specific other apparatus within a predetermined period set in advance for determination When there is a change in the connection state of the network 9, such as when a reply command is not returned or when a new information processing apparatus is connected to the network 9, the information is notified to the ability exchange program described later. .

  When the main processor 21 receives an inquiry from another manager on the network 9 and a notification of completion of setting the MS status of the own apparatus from the MS manager, the main processor 21 executes the capability exchange program.

  When the own device is a master device, the capability exchange program acquires device information of all other devices connected to the network 9, that is, device information of each slave device. As described above, the device information of another device is generated by generating a software cell in which the DMA command is a status request command and transmitting it to the other device. Thereafter, the DMA command is a status return command and data of the other device. This is possible by receiving a software cell containing device information from another device.

  Similar to the device information table of the own device that is the master device, the capability exchange program sets an area for storing device information of all other devices (each slave device) connected to the network 9 as the main memory of the own device. This information is recorded in a device information table of another device (slave device). That is, the device information of all information processing devices connected to the network 9 including the device itself is recorded in the main memory 26 of the master device as a device information table.

  On the other hand, when the own device is a slave device, the capability exchange program acquires device information of all other devices connected to the network 9, that is, device information of each slave device other than the master device and the own device. The information processing apparatus ID and the MS status included in the apparatus information are recorded in the main memory 26 of the own apparatus. That is, the device information of the own device is recorded as a device information table in the main memory 26 of the slave device, and the master device connected to the network 9 other than the own device and the information processing device ID for each slave device. And the MS status are recorded as another device information table.

  Further, in both the master device and the slave device, when the capability exchange program is notified from the MS manager that the information processing device is newly connected to the network 9 as described above, the device of the information processing device Information is acquired and recorded in the main memory 26 as described above.

  Note that the MS manager and the capability exchange program are not limited to being executed by the main processor 21, but may be executed by any of the sub processors 23. The MS manager and the capability exchange program are preferably resident programs that always operate while the main power supply of the information processing apparatus is turned on.

  For both the master device and the slave device, the capability exchange program causes the MS manager to cut off the main power supply of the information processing device connected to the network 9 or disconnect the information processing device from the network 9 as described above. When it is notified, the apparatus information table of the information processing apparatus is deleted from the main memory 26 of the own apparatus.

  Further, when the information processing apparatus disconnected from the network 9 is a master apparatus, a new master apparatus is determined by the following method.

  Specifically, for example, each of the information processing apparatuses that are not disconnected from the network 9 replaces the information processing apparatus ID of the own apparatus and the other apparatus with a numerical value, and sets the information processing apparatus ID of the own apparatus to the information processing of the other apparatus. If the information processing device ID of the own device is the smallest among the information processing devices not disconnected from the network 9 as compared with the device ID, the slave device moves to the master device and sets the MS status to 0. As described above, device information of all other devices (each slave device) connected to the network 9 is acquired and recorded in the main memory 26 as a master device.

  As shown in the lower part of FIG. 5, in order for a plurality of information processing devices 1, 2, 3, 4 connected to the network 9 to operate as a single virtual information processing device 7, the master device is a user. It is necessary to grasp the operation of the slave device and the operating state of the slave device.

  FIG. 7 shows a state in which four information processing apparatuses operate as one virtual information processing apparatus 7. It is assumed that the information processing device 1 is operating as a master device, and the information processing devices 2, 3, and 4 are operating as slave devices A, B, and C.

  When the user operates an information processing device connected to the network 9, if the operation target is the master device 1, the operation information is directly grasped by the master device 1, and if the operation target is a slave device, The operation information is transmitted from the operated slave device to the master device 1. That is, regardless of whether the user's operation target is the master device 1 or the slave device, the operation information is always grasped by the master device 1. The operation information is transmitted, for example, by a software cell whose DMA command is an operation information transmission command.

  Then, the main processor 21-1 included in the information processing controller 11 in the master device 1 selects a function program to be executed according to the operation information. At that time, if necessary, the main processor 21-1 included in the information processing controller 11 in the master device 1 may transfer the main memory 26-1 from the external recording units 28-1 and 28-2 of the own device by the above method. However, another information processing device (slave device) may transmit the function program to the master device 1.

  In the function program, information processing device type IDs represented as information shown in FIG. 4, main processor or sub-processor processing capacity, main memory usage, and conditions related to the external recording unit are required for each execution unit. The required specifications regarding the device are defined.

  The main processor 21-1 included in the information processing controller 11 in the master device 1 reads out the required specifications necessary for each function program. Further, the device information table of each information processing device is read by referring to the device information table previously recorded in the main memory 26-1 by the capability exchange program. The device information here is information related to the main processor, sub-processor, main memory, and external recording unit.

  The main processor 21-1 included in the information processing controller 11 in the master device 1 sequentially compares the device information of each information processing device connected on the network 9 with the required specifications necessary for executing the function program. To do.

  For example, when the function program requires a recording function, only the information processing apparatus having the recording function is specified and extracted based on the information processing apparatus type ID. Furthermore, a slave device that can secure the conditions regarding the processing capability of the main processor or sub processor, the amount of main memory used, and the external recording unit necessary for executing the function program is specified as an execution request candidate device. Here, when a plurality of execution request candidate devices are specified, one execution request candidate device is specified and selected from the candidate devices.

  When the slave device to be executed is specified, the main processor 21-1 included in the information processing controller 11 in the master device 1 sets the main memory included in the information processing controller 11 in the own device for the specified slave device. The device information table of the slave device recorded in 26-1 is updated.

  Further, the main processor 21-1 included in the information processing controller 11 in the master device 1 generates a software cell in which the DMA command is a function program execution command, and a necessary sub-routine related to the function program is provided to the cell interface of the software cell. The processor information and the sandbox size (see FIG. 3) are set and transmitted to the slave device requested to execute.

  The slave device requested to execute the function program executes the function program and updates the device information table of the own device. At that time, if necessary, the main processor 21 included in the information processing controller in the slave device, from the external recording unit 28 of the own device to the main memory 26 by the above method, the function program and the sub-operation that operates in cooperation with the function program. Load the processor program.

  When the required function program or the sub processor program that operates in cooperation with the function program is not recorded in the external recording unit 28 of the slave device requested to execute the function program, the other information processing apparatus Alternatively, the system may be configured so that the sub processor program is transmitted to the function program execution request destination slave device.

  The sub-processor program can be executed by another information processing apparatus using the aforementioned load command and kick command.

  After the execution of the function program, the main processor 21 included in the information processing controller in the slave device that has executed the function program transmits an end notification to the main processor 21-1 included in the information processing controller 11 in the master device 1. At the same time, the device information table of the own device is updated. The main processor 21-1 included in the information processing controller 11 in the master device 1 receives the end notification and updates the device information table of the slave device that has executed the function program.

  The main processor 21-1 included in the information processing controller 11 in the master device 1 identifies itself as an information processing device that can execute the function program from the reference result of the device information table of the own device and the other device. There is also a case of selecting. In that case, the master device 1 executes the function program.

  In the example of FIG. 7, the case where the user operates the slave device A (information processing device 2) and another slave device B (information processing device 3) executes a function program according to the operation is described with reference to FIG. An example of the distributed processing will be described.

  In the example of FIG. 8, when the user operates the slave device A, distributed processing of the entire network system including the slave device A starts. First, in step 81, the slave device A transmits the operation information to the master device. 1 to send.

  In step 72, the master device 1 receives the operation information, and further proceeds to step 73, where each information processing is performed from the device information table of the own device and other devices recorded in the main memory 26-1 of the own device. The operating state of the apparatus is checked, and an information processing apparatus that can execute a function program corresponding to the received operation information is selected. This example is a case where the slave device B is selected.

  Next, in step 74, the master device 1 requests the selected slave device B to execute the function program.

  In step 95, the slave device B receives the execution request, and further proceeds to step 96 to execute the function program requested to be executed.

  As described above, by operating only one information processing apparatus, the user operates a plurality of information processing apparatuses 1, 2, 3, and 4 without operating other information processing apparatuses. The information processing apparatus 7 can be operated.

  Next, an embodiment of the present invention will be described on the premise of the network system as described above.

  FIG. 9 is a block diagram showing a configuration of a communication system according to an embodiment of the present invention. In the communication system 300, for example, a transmission device 100 and a reception device 200 are connected via a network 50. The network 50 is configured by, for example, the Internet or a LAN (Local Area Network).

  For example, the transmission device 100 and the reception device 200 each include at least the functions of the information processing device 1 and the like, and are connected to each other via the network 50. The communication system 300 realizes a system such as remote video conference, Internet telephone, video stream distribution, and digital broadcasting. Specifically, if the system is a system such as video stream distribution, the transmission device 100 is, for example, a video stream distribution device, and the reception device 200 is a reception terminal device used by a user. Alternatively, when the network 50 is a home LAN, for example, the transmission device 100 is a home server connected to a WAN (Wide Area Network), and the reception device 200 provides video and audio from the home server. It is a receiving terminal device.

  The transmission device 100 is connected to an image and audio input unit 12 including, for example, a camera 12a and a microphone 12b. The transmission device 100 takes in the data input from the input unit 12. On the other hand, to the receiving apparatus 200, for example, an output unit 13 including a monitor 13a and a speaker is connected. The receiving device 200 outputs video and audio from the data received from the transmitting device 100 via the output unit 13.

  The transmission apparatus 100 includes a system control unit 31, a plurality of (for example, three) encoders 39a, 39b, and 39c, a packet generation unit 32, a packet analysis unit 33, a network monitoring unit 34, an RTCP (RTP Control Protocol) packet generation unit 35, It has a time generator 36, an RTP (Real Time Transport Protocol) output port 37, an RTCP port 38, and a TCP (Transmission Control Protocol) port 40.

  Here, the “packet” has the same concept as the “software cell” (see FIG. 3) described above, and hereinafter, the “packet” may be read as a “software cell”. Alternatively, a collection of several packets may constitute a software cell.

  The system control unit 31 controls the transmission apparatus 100 as a whole. For example, the main processor 21-1 (which the information processing apparatus 1 shown in FIG. 1 has) mainly functions as the system control unit 31. Fulfill.

  The first encoder 39a, the second encoder 39b, and the third encoder 39c compress and encode the data input from the input unit 12 using different codecs. Specifically, as shown in FIG. 10, the first encoder 39a uses the first codec 15a, the second encoder 39b uses the second codec 15b, and the third encoder 39c uses the third codec. 15c is used. If the transmission apparatus 100 is the information processing apparatus 1 shown in FIG. 1, the sub-processors 23-1, 23-2, 23-3 and the like are assigned as the encoders 39a, 39b and 39c, respectively. For example, a DSP (Digital Signal Processor) is used as these sub-processors. The system control unit 31 may control the codec allocation according to the network status, the load status of the transmission device 100, and the like. Specifically, for example, one codec can be used by using three sub-processors 23-1, 23-2, and 23-3. The codec referred to here corresponds to the above-described sub processor program or function program.

  The first, second, and third codecs 15a, 15b, and 15c are different codecs as described above. Here, different codecs are, for example, algorithms, programs, or the same even if the standards and formats are the same, such as MPEG (Moving Picture Experts Group), WMV (Windows (registered trademark) Media Video), or H.261. If the program version is different, it is called a different codec. The same applies to the audio codec.

  The packet generator 32 generates a media packet according to RTP using media data such as video and audio as a payload. That is, the RTP header is added to the payload data and packetized. The media packet generated by the packet generator 32 is a packet having a plurality of continuity. The transmission delay time of data in the network 50 can be set by using media packets having such continuity. Further, the packet generator 32 generates a TCP packet for exchanging capabilities with the receiving device 200, for example.

  For example, the RTCP packet generation unit 35 generates an Echo-Reply packet conforming to RTCP for causing the reception device 200 to measure the data transmission delay time under a predetermined condition described later. In addition, the RTCP packet generator 35 generates an RTCP packet having, for example, an EOS (End Of Stream) message, and sends the packet to the receiving device 200 to clearly indicate the end of the data stream.

  The packet analysis unit 33 analyzes the type of packet and data included in the packet.

  The time generation unit 36 generates a reference clock for the system control unit 31 and gives a time stamp to the packet generation unit 32 and the RTCP packet generation unit 35 based on the generated clock.

  The TCP port 40 is an input / output port for performing communication according to TCP via the network 50. Specifically, the TCP port 40 functions as a port for exchanging status information at least during the capability exchange between the transmission device 100 and the reception device 200.

  For example, the network monitoring unit 34 acquires the load status and delay status of the network 50. Specifically, information on the packet loss rate and delay time measured by the receiving device 200 is acquired from the receiving device 200 via the network 50. In addition, the network monitoring unit 34 acquires status information including capability information and setting information of the receiving device 200.

  The configuration of the RTP packet is shown in FIG. The RTP header includes version number (v), padding (P), presence / absence of extension header (X), number of transmission sources (Counter), marker information (marker bit), payload type (Payload type), sequence number, time stamp , Fields of synchronization source (source) identifier (SSRC) and contributing source (source) identifier (CSRC) are provided. In the case of a software cell, the sequence number is the ID (global ID) of the software cell shown in FIG. On the data receiving side, the processing time is controlled when the RTP packet is expanded based on the time stamp added to the RTP header, thereby enabling real-time image or audio reproduction control. For example, in an RTP packet storing encoded data of moving image data, a common time stamp is set for a plurality of RTP packets belonging to one image frame, and a termination packet constituting each frame is a termination. An identification flag is stored in the RTP header. Packets to which the RTP header is added are further communicated on the network 50 with, for example, a UDP (User Datagram Protocol) header or an IP (Internet Protocol) header. In this case, for example, a packet to which such a UDP header or IP header is added becomes the above-described software cell.

  The receiving apparatus 200 includes an RTP input port 51, a packet analysis unit 52, a packet processing unit 53, an RTCP port 54, a network monitoring unit 55, a delay measurement unit 56, a database 57, a system control unit 58, and a plurality of (for example, three) decoders. 59a, 59b, 59c, and TCP port 60.

  The system control unit 58 generally controls the receiving device 200 as a whole. For example, the main processor 21-2 (which the information processing device 2 shown in FIG. 1) mainly has the function of the system control unit 31. Fulfill.

  The packet analysis unit 52 analyzes an RTP packet (for example, a media packet) input from the RTP input port, and analyzes an RTCP packet input / output at the RTCP port 54. Specifically, the IP header and RTP header for each layer are removed, and the data contained in the packet, such as the packet type, is analyzed.

  The packet processing unit 53 processes the packet data acquired from the packet analysis unit 52, passes the data to the first, second, and third encoders 59a, 59b, and 59c, or sends an Echo packet for delay measurement according to RTCP. Or generate. Further, as will be described later, when the network monitoring unit 55 detects a loss or error of a packet transmitted from the transmission device 100, the packet processing unit 53 executes a packet retransmission request that has resulted in the loss or error. It is determined whether or not to perform (ARQ (Auto Repeat reQuest)). When executing a packet retransmission request, the packet processing unit 53 generates a retransmission request packet that stores data for causing the transmission apparatus 100 to perform retransmission of the lost packet detected by the packet analysis unit 52. In addition, the packet processing unit 53 generates a reception confirmation response packet when a certain predetermined number of packets are successfully received.

  For example, the network monitoring unit 55 acquires the load status of the network 50, or acquires status information including the capability and setting information of the transmission device 100 via the network 50.

  The delay measuring unit 56 measures the round-trip transmission delay time (RTT) or one-way transmission delay time of data in the network 50 based on the RTP packet or the RTCP packet, and stores the measured data in the database 57. If the receiving device 200 is the information processing device 2 shown in FIG. 1, the database 57 corresponds to, for example, the main memory 26-2, the external recording unit 28-3, and the like.

  The first decoder 59a, the second decoder 59b, and the third decoder 59c decode the data processed by the packet processing unit 53 using different codecs and output the decoded data to the output unit 13. Specifically, as shown in FIG. 11, the first decoder 59a uses the first codec 15a, the second decoder 59b uses the second codec 15b, and the third decoder 59c uses the third codec 15a. The codec 15c is used. Similarly to the case described with reference to the transmission apparatus 100, if the reception apparatus 200 is the information processing apparatus 2 shown in FIG. 1, the decoders 59a, 59b and 59c may be sub-processors 23-4, 23-5, 23, for example. -6 etc. (see FIG. 1) are assigned, respectively, and the system control unit 58 can also variably control the codec assignment.

  FIG. 13 shows the configuration of an Echo packet according to RTCP, and FIG. 14 shows the configuration of an Echo-Reply packet. As shown in FIG. 13, the Echo packet stores a header (HEAD), format (FORMAT), packet type, packet length, transmission synchronization source identifier (RTCP), and ECHO-ID as identification data of the Echo packet. As shown in FIG. 14, the Echo-Reply packet includes a header (HEAD), a format (FORMAT), a packet type, a packet length, a transmission synchronization source identifier (RTCP), an ECHO-ID corresponding to the ECHO-RTCP packet, and device processing. Time (delay due to apparatus processing time described later) is stored. By inserting the ECHO-ID in the Echo packet and the Echo-Reply packet, when the transmitting apparatus 100 receives the Echo-Reply packet, it is possible to identify when the reply (Echo-Reply) corresponds to the transmitted Echo packet. It becomes possible.

  The transmission device 100 and the reception device 200 described above may include all the functions of the other party. The status information is executed by TCP, but may be another protocol such as RTP.

  The operation of the communication system 300 configured as described above will be described. FIG. 15 is a flowchart showing the operation.

  The transmission device 100 acquires the status information of the reception device 200 connected to the network 50 using the capability exchange program described above (step 1501). Specifically, the transmission device 100 generates a TCP packet of a status request command including the device ID of the transmission device 100 as a response destination ID (see FIG. 3) and the device ID of the reception device 200 as a transmission destination ID. 32 is generated, transmitted to the receiving apparatus 200, and executed. The receiving device 200 that has received the TCP packet of the status request command generates a TCP packet of the status reply command by the packet processing unit 53 and transmits its own status information to the transmitting device 100.

  Conversely, the receiving apparatus 200 may issue a status request command, and the transmitting apparatus 100 may transmit its own status information to the receiving apparatus 200 in response to this.

  FIG. 16 shows the structure of the data area of the status reply command packet generated by the receiving apparatus 200, for example. Here, “packet loss rate”, “round trip transmission delay time (RTT)”, “codec information”, and the like are further added to the data shown in FIG.

  The packet loss rate is a ratio of the number of packets lost on the network 50 within a certain time, for example, among the number of packets transmitted from the transmission device 100. Since a sequential number (for example, see FIG. 12) is added to each packet, the receiving apparatus 200 can recognize a lost packet from this number.

  There are roughly two types of packet loss: random loss and burst loss. The random loss is generated for each device depending on the performance of the transmission device 100, the reception device 200, or a node (router or the like) in the network 50, and is about several percent. On the other hand, the burst loss occurs when the bandwidth is compressed due to congestion (congestion) of the network, and may reach, for example, 10 to 30% in a short time. The receiving device 200 periodically sends such a packet loss rate to the transmitting device 100, for example, so that the transmitting device 100 grasps the loss rate by the network monitoring unit 34 and a random loss is currently occurring. Or whether a burst loss has occurred.

  RTT means that after the receiving apparatus 200 transmits an RTCP Echo packet, the transmitting apparatus 100 receives the Echo packet, generates an Echo-Reply packet, and transmits the Echo-Reply packet to the receiving apparatus 200. The receiving apparatus 200 transmits the Echo packet. -Reply time until receiving a packet.

  The codec information includes, for example, information such as bit rate information in addition to the above-described codec type. The codec information included in the status information transmitted for the first time in the initial state where the connection between the transmission device 100 and the reception device 200 is established is the codec information set for the reception device 200 by the user.

  The other setting information is, for example, setting information such as which one of image quality, sound quality, network bandwidth, etc. is to be prioritized. This is user setting information transmitted in the first status information. is there. Therefore, unless the user sets again thereafter, the other setting information is not transmitted after the second time.

  When the transmission apparatus 100 acquires such status information from the reception apparatus 200, the transmission apparatus 100 can grasp the capability and setting information of the reception apparatus 200, that is, various types of information illustrated in FIG. Status information is transmitted and received periodically, for example, every 100 ms or longer. In the first transmission / reception of the status information after the connection is established, the packet loss rate is not included in the status information of the receiving device 200. This is because the receiving apparatus 200 has not yet received a packet from the transmitting apparatus 100.

  Returning to the flow of FIG. 15, when the transmitting apparatus 100 acquires status information, the transmitting apparatus 100 selects an optimal communication mode based on the status information in the initial state (step 1502). The optimal communication mode here is included in the status information by using as much as possible the processor to be used, the number of processors, or the memory according to the current capability of the transmission apparatus 100 and the load status. It operates so as to satisfy the codec, bit rate, etc. required by the receiving apparatus 200. The current capability and load status of the transmission device 100 are, for example, the status of the main processor, the usage rate of the sub processors, the remaining amount of memory, and the like. That is, in the initial state, the transmission device 100 cannot recognize the network status, and therefore attempts to communicate by recognizing the capability status of the transmission device 100.

  FIG. 17A shows codec information and other setting information set by the user among the status information of the receiving apparatus 200. In this example, the receiving apparatus 200 receives a moving image. For example, it is assumed that the receiving apparatus 200 uses the first codec 15a, the set bit rate is a maximum of 8.0 Mbps, and the priority parameter is the image quality. Then, the transmission apparatus 100 encodes data using the first codec 15a, that is, the first encoder 39a, and starts transmission to the reception apparatus 100 so as to correspond thereto. At this time, the transmitting apparatus 100 encodes data at a maximum bit rate of 8.0, which is a bit rate adapted to the first encoder. That is, as described above, the transmission device 100 operates to satisfy the request of the reception device 200 shown in FIG. 17A as much as possible using the plurality of sub-processors 23 while determining its own load status (step 1503). .

  During the data transmission, the transmission device 100 and the reception device 200 monitor the status of the network 50 by the network monitoring units 34 and 55 (step 1504). For example, the transmission device 100 acquires and updates new status information from the reception device 200, generates an RTCP packet, and transmits the RTCP packet to the reception device 200. The receiving apparatus 200 also measures the packet loss rate so as to correspond to the operation of the transmitting apparatus 100, or generates an RTCP packet and transmits it to the transmitting apparatus 100. The reason why the RTCP packet is used is that the lost packet is retransmitted by the ARQ system or the transmission delay time is measured as described later. Update of status information and measurement of transmission delay time are performed periodically, for example, every few seconds to several tens of seconds, or every few minutes, but are not limited thereto.

  Then, the transmission apparatus 100 and the reception apparatus 200 perform communication by resetting the codec or resetting the bit rate based on the acquired status information or based on the transmission delay time measured by the reception apparatus 200. (Step 1505).

  FIG. 18 is a flowchart showing an operation example of the receiving apparatus 200 in the above steps 1504 and 1505. The receiving apparatus 200 calculates the packet loss rate to be included in the status information as one parameter for monitoring the network 50 by the network monitoring unit 55 (step 1801). After calculating the packet loss rate, the network monitoring unit 55 determines that a random loss has occurred if the loss rate is less than 10%, for example (NO in step 1802). On the other hand, if the loss rate is 10% or more and is calculated continuously several times (for example, 3 to 5 times), it is determined that a burst loss has occurred (YES in step 1802).

  In the case of random loss, the system control unit 31 performs control so as to make a retransmission request for a packet lost due to the ARQ system (step 1803). The ARQ algorithm may be a general one. When a random loss has occurred, the bandwidth of the network 50 is not compressed as much as when a burst loss has occurred, so there is no problem even if a retransmission request is made in this way.

  On the other hand, when it is determined by the network monitoring unit 55 that a burst loss has occurred, the system control unit 31 performs, for example, a second corresponding to a bit rate of 1.5 Mbps as shown in FIG. Control to switch to the codec 15b. This is a lower bit rate than that used in the first codec. Specifically, after YES in step 1802, the receiving apparatus 200 rewrites the codec information in the status information to “second codec” and transmits this status information to the transmitting apparatus 100 (step 1804).

  FIG. 19 is a flowchart showing the operation of the transmission apparatus 100 after step 1804. The transmission device 100 receives the status information transmitted from the reception device 200 (step 1901). Then, the transmitting apparatus 100 reads the status information and switches the codec that is the updated part from the first codec 15a to the second codec 15b (step 1902). Thereafter, the transmitting apparatus 100 encodes the data with the switched second codec 15b and transmits the data to the receiving apparatus 200 (step 1903). Thereafter, the receiving apparatus 200 receives the transmitted data, decodes it by the second decoder 59b, and reproduces it.

  As described above, in the present embodiment, the optimal one of the first, second, and third codecs 39a, 39b, and 39c depends on the time zone and the network conditions that change depending on the receiving apparatus 200 that is the connection partner. Since the codec is selected, the receiving apparatus 200 can always receive data with the optimum image quality and sound quality. Conventionally, communication has been performed with a variable bit rate. However, even when a high bit rate codec is used when decoding a receiving apparatus, the frame rate is lowered, resulting in awkward communication. However, according to the present embodiment, since each codec is variable, it is possible to communicate smoothly.

  Further, for example, by switching from a codec for high bit rate to a codec for low bit rate, the load on the processor can be reduced. In particular, when the computer system that operates in cooperation described with reference to FIGS. 1 to 8 is employed, a plurality of processors are often used when the network 50 is congested, and the apparatus load may be increased. Therefore, it is very effective if the load on the processor can be reduced according to the network conditions.

  Further, when there are a plurality of codecs, if one processor is used for operation, problems such as load fluctuations and unexpected operations occur, and it is necessary to consider a complicated situation. However, the allocation process can be simplified by using a plurality of processors as in the present embodiment.

  When no packet loss has occurred, the transmission device 100 and the reception device 200 continue communication with the codec and bit rate set first. In addition, when the loss rate of the burst loss is larger, it may be switched to the third codec 15c for a lower bit rate. Then, when the loss rate becomes lower than a predetermined value thereafter, it is possible to switch to the first or second codec that supports a higher bit rate than the third codec.

  Of the processes described in FIG. 18, the processes in steps 1801 and 1802 may be executed by the transmission apparatus 100. In this case, after YES in step 1802, the transmitting apparatus 100 executes the processing from step 1902 onward, and the receiving apparatus 200 decodes with the switched codec.

  Furthermore, the receiving device 200 may send only the information about random loss or burst loss to the transmitting device 100 instead of the packet loss rate information in the status information.

  Next, the round trip transmission delay time (RTT) will be described.

  The receiving apparatus 200 calculates, for example, a round trip transmission delay time (RTT) periodically or at an arbitrary timing. FIG. 20 is an Echo packet transmission process flow. The transmission of the Echo-Reply packet can be actively executed by the receiving apparatus 200 periodically or at an arbitrary timing. If an RTT measurement request occurs in the RTCP packet transmission standby state in step 2001, the process proceeds to the Echo packet generation process in step 2002. The Echo packet generated in step 2002 has the configuration described above with reference to FIG. ECHO-ID as an identifier unique to each packet is set in the packet. In step 2003, the generated Echo packet is transmitted to the transmission device 100 via the RTCP port 38. In step 2004, the transmission time when the Echo packet is transmitted and the ECHO-ID set in the packet are recorded in the database 57.

  FIG. 21 is an Echo-Reply packet reception and RTT calculation processing flow. If it is determined in step 2101 that the Echo-Reply packet has been received in the RTCP packet reception standby state (YES in step 2102), the reception time of the Echo-Reply packet is recorded in the database 57 in step 2103. The received Echo-Reply packet has the configuration described above with reference to FIG. The packet is set with ECHO-ID as the same identifier as the corresponding Echo packet that is the response target, and further stores the device processing time calculated by the transmitting device 100. The apparatus processing time is the time from when the transmitting apparatus 100 receives an Echo packet until it transmits an Echo-Reply packet.

  In step 2104, the apparatus processing time is extracted from the received Echo-Reply packet, and in step 2105, the transmission time of the corresponding Echo packet is searched from the ECHO-ID set in the received Echo-Reply packet.

In step 2106, RTT calculation processing is executed based on the reception time of the Echo-Reply packet acquired in step 2103, the apparatus processing time acquired in step 2104, and the transmission time of the Echo packet acquired in step 2103. The RTT is calculated based on the following equation (1).
RTT = (Echo-Reply packet reception time) − (Echo packet transmission time) − (device processing time) Equation (1).

  The RTT calculated by the above formula is transmitted as status information from the receiving apparatus 200 to the transmitting apparatus 100 as shown in FIG. Thus, the transmission apparatus 100 can grasp the network status by referring to the RTT by the network monitoring unit 34. For example, when the RTT is relatively large, there is a high possibility that packet burst loss has occurred in the network 50. For example, when a situation occurs in which the RTT of 10 ms to 30 ms exceeds 100 ms, a burst loss will occur in the network 50 or a burst loss is currently occurring. Therefore, by measuring the RTT, the receiving device 200 can execute the processing after step 1802 shown in FIG. 18, and then the transmitting device 100 can execute the processing shown in FIG. .

In the above description, the case of using the RTT has been described. However, the transmission device 100 or the reception device 200 may measure the one-way transmission delay time of the network 50 instead of or in addition to the RTT. Specifically, when the reception device 200 calculates, it is calculated by the following equation (2).
One-way delay = (packet reception interval of reception device) − (transmission interval of packet of transmission device) Equation (2).

  FIG. 22 is a diagram showing a configuration of a communication system according to another embodiment of the present invention.

  This communication system is a system for realizing a video conference, for example. This communication system includes an MCU server 400 on which an MCU is mounted, and a plurality of terminal devices 500a, 500b, 500c, and 500d connected to the MCU server 400 via a network 50. The MCU server 400 (hereinafter simply referred to as a server) basically has the configuration of the transmission device 100 shown in FIG. Each of the terminal devices 500a to 500d basically has the configuration of the receiving device 200 shown in FIG. 9, and each of the terminal devices 500a to 500d has the input unit 12 having the camera 12a and the like shown in FIG. And the output part 13 which has the monitor 13a etc. is connected.

  In addition, the server 400 includes an image composition processing unit (not shown) that performs composition processing on each image in order to receive each image received from each of the terminal devices 500a to 500d and display it on one screen, for example. Thereby, the server 400 can distribute the synthesized image to each of the terminal devices 500a to 500d by multicast or the like. In the case of this example, since there are four terminal devices, for example, one screen is distributed as one image in the form of being divided into four equal parts. If this communication system is compared to the network system shown in FIGS. 1 to 8, the server 400 is a master device, and the terminal devices 500a to 500d are slave devices.

  FIG. 23 shows the codec that each of the server 400 and the terminal devices 500a to 500d has as a premise of the present embodiment. As shown in the figure, it is assumed that the server 400 includes first to third codecs 15a to 15c. The terminal device 500a includes a first codec 15a and a third codec 15c, and the terminal device 500b similarly includes a first codec 15a and a third codec 15c. The terminal device 500c includes a second codec 15b and a third codec 15c, and the terminal device 500d includes a third codec 15c and a fourth codec 15d. The fourth codec 15d is different from the first, second, and third codecs.

  The server 400 includes three encoders (three subprocessors) 39a to 39c that use the first to third codecs 15a to 15c (see FIG. 9).

  Hereinafter, the operation of this communication system will be described. FIG. 24 is a flowchart showing the operation until the connection between the server 400 and each of the terminal devices 500a to 500d is confirmed in this communication system.

  When only the server 400 is initially connected to the network 50, the server 400 is in a connection request waiting state for each of the terminal devices 500a to 500d (step 2401). When the terminal devices 500a to 500d are connected to the network 50 (step 2402), the server 400 and the terminal devices 500a to 500d exchange status information as described above (step 2403). Here, the status information exchanged from each of the terminal devices 500a to 500d with the server 400 is, for example, data having contents shown in FIG.

  The server 400 confirms the number of connected terminal devices, and if there are more than one, proceeds to Step 2405 (YES in Step 2404). In the present embodiment, since four terminal devices are connected, the process proceeds to step 2405. When the number of terminal devices to be confirmed is one, a one-to-one connection is started between the server 400 and one terminal device, and packets are exchanged (step 2406). Step 2406 can be processed according to the flow described in FIG.

  In step 2405, the server 400 confirms whether there is a terminal device corresponding to the MCU, and if not, a connection error occurs (step 2407). If there is a terminal device corresponding to the MCU, the server 400 starts multipoint connection (step 2408). In this embodiment, the four terminal devices 500a to 500d will be described as terminal devices corresponding to MCUs.

  FIG. 25 is a flowchart showing the operation of the server 400 after step 2408.

  Based on the status information received from each of the terminal devices 500a to 500d, the server 400 determines whether at least two of the terminal devices 500a to 500d use different codecs or not (step 2501). When all of the terminal devices 500a to 500d use, for example, the third codec 15c, which is the codec that is initially set by the user, for example (NO in step 2501), the process proceeds to step 2503. Otherwise, as an example, as shown in FIG. 26, when the terminal devices 500a and 500b use the first codec 15a, the terminal device 500c uses the second codec 15b, and the terminal device 500d uses the third codec 15c, 400 proceeds to step 2502.

  In step 2502, whether or not the server 400 can allocate a plurality of processors (CPUs) of the server 400 for each codec, that is, whether or not there is room to use all of the plurality of processors of the server 400 for each codec. Check. The reason for this confirmation is that when the number of terminal devices is large and each of these terminal devices uses a different codec, the number of processors even if the server 400 has a large number of codecs for those This is because a processor cannot be assigned to each codec if the number of codecs is less than the number of codecs.

  The server 400 includes three encoders 39a to 39c that use the first to third codecs 15a to 15c. Therefore, if these encoders (sub-processors) are not used in other processes, in step 2502, the server 400 can assign all sub-processors to all codecs. In this case, the server 400 assigns the first codec 15a used by the terminal devices 500a and 500b to the first encoder 39a, and assigns the second codec 15b used by the terminal device 500c to the second encoder 39b (step 2503). .

  Thereafter, the server 400 encodes the data using each set codec and bit rate, and starts transmission of the encoded data (step 2504). During the data transmission, the server 400 executes the same processing as the flow shown in FIG. 19 for each of the terminal devices 500a to 500d (step 2505) (in this case, each of the terminal devices 500a to 500d 18).

  If it is not possible to assign each codec in step 2502, the process proceeds to step 2506 (NO in step 2502). The case where allocation for each codec is impossible is as follows. For example, when each of the terminal devices 500a to 500d uses the codec shown in FIG. 26, the server 400 cannot use one or two of the three codecs. This is because the server 400 is for some reason, for example, when the server 400 is using a sub-processor in a process other than the MCU system.

  Another reason for the case where allocation for each codec is impossible is when the terminal device 500d uses the fourth codec 15d that the server 400 does not have. That is, the server 400 recognizes in step 2403 that the initial setting of the terminal device 500d is set to use the fourth codec 15d.

  In such a case, when the number of assignable sub-processors, that is, the number of assignable codecs is one (YES in step 2506), the server 400 can handle all the terminal devices 500a to 500d with one codec. Switch to the correct codec (step 2507). In this case, the codec is the third codec 15c. Accordingly, the server 400 can encode data using the third codec 15c and start transmission (step 2504). Thereby, each terminal device 500a-500d can each decode the data transmitted using the 3rd codec 15c. In this case, in step 2505, the server 400 switches the codec in accordance with the network status or the like. However, if the number of sub-processors that can be allocated at that time is still one, the server 400 Is not switched.

  In step 2506, when the number of sub processors that can be allocated, that is, the number of codecs that can be allocated is two, the server 400 proceeds to step 2508 (NO in step 2506). In step 2508, the third codec 15c, which is a codec that can be used for all the terminal devices 500a to 500d, and the common codec used by the terminal device having the second largest number of terminal devices using the third codec 15c, that is, the first codec. 1 codec 15a is assigned (see FIG. 26). Accordingly, the terminal devices 500a and 500b can decode data transmitted using the first codec 15a, and the terminal devices 500c and 500d can decode data transmitted using the third codec, respectively.

  As described above, server 400 according to the present embodiment can assign a codec based on the acquired status information for each terminal device 500a to 500d. Thereby, even if codecs differ for every terminal device 500a-500d, it can communicate in an optimal environment. In addition, the load on the server 400 can be reduced by assigning codecs according to the load status of the server 400 or according to the server capabilities.

It is a figure which shows an example of the network system used as the premise in this invention. It is a figure with which it uses for description of the information processing controller with which an information processing apparatus is provided. It is a figure which shows an example of a software cell. It is a figure which shows the data area of a software cell when a DMA command is a status reply command. It is a figure which shows a mode that a some information processing apparatus operate | moves as one virtual information processing apparatus. It is a figure which shows an example of the software configuration of an information processing controller. It is a figure which shows a mode that four information processing apparatuses operate | move as one virtual information processing apparatus. It is a figure which shows the example of the distributed process in the system of FIG. It is a block diagram which shows the structure of the communication system which concerns on one embodiment of this invention. It is a figure which shows the codec of the encoder which a transmitter has. It is a figure which shows the codec of the decoder which a receiver has. It is a block diagram of a RTP packet. It is a block diagram of an Echo packet. It is a block diagram of an Echo-Reply packet. It is a flowchart which shows operation | movement of the said communication system. It is a figure which shows the structure of the data area | region of the packet of the status reply command which a receiver produces | generates. (A) and (b) respectively indicate codec information and other setting information in the status information. 16 is a flowchart illustrating an operation example of the reception device in steps 1504 and 1505 illustrated in FIG. 15. It is a flowchart which shows operation | movement of the transmitter after step 1804 shown in FIG. It is a figure which shows the transmission processing flow of an Echo packet. It is a figure which shows the reception of an Echo-Reply packet, and a RTT calculation processing flow. It is a block diagram which shows the structure of the communication system which concerns on other embodiment of this invention. It is a figure which shows the codec which a server and each terminal device have, respectively. It is a flowchart which shows operation | movement until the connection of a server and each terminal device is confirmed. It is a flowchart which shows operation | movement of the server after step 2408 shown in FIG. It is a figure which shows the codec which each terminal device uses in an initial state.

Explanation of symbols

1, 2, 3 ... Information processing device 9, 50 ... Network 15a ... First codec 15b ... Second codec 15c ... Third codec 15d ... Fourth codec 34, 55 ... Network monitoring unit 39a ... First Encoder 39b ... second encoder 39c ... third encoder 55 ... network monitoring unit 59a ... first decoder 59b ... second decoder 59c ... third decoder 100 ... transmitting device 200 ... receiving device 300 ... communication system 400 ... MCU server 500a-500d ... Terminal device

Claims (16)

A first encoder that encodes data using a first codec;
A second encoder that encodes the data using a second codec different from the first codec;
Transmitting means capable of transmitting data encoded by the first and second encoders to a receiving apparatus connected via a network;
Monitoring means for monitoring the status of the network;
Selecting means for selecting which of the first and second encoders the data is to be encoded and transmitted by the transmission means according to at least the status of the monitored network; A transmitter characterized by the above. The transmission device according to claim 1,
The monitoring means includes means for obtaining a loss rate of data transmitted on the network;
The selection means selects which of the first and second encoders encodes the data and transmits the data by the transmission means according to the acquired loss rate. apparatus. The transmission device according to claim 1,
The monitoring means includes means for detecting whether a loss of data transmitted on the network is a random loss or a burst loss;
The transmission device according to claim 1, wherein the selection unit selects which of the first and second encoders encodes the data and transmits the data by the transmission unit according to the detection result. The transmission device according to claim 3,
The transmission apparatus further comprising means for transmitting data lost due to the random loss to the receiving apparatus when the monitoring means detects the random loss. The transmission device according to claim 1,
Further comprising means for acquiring status information including at least one of the capabilities and settings of the receiving device from the receiving device via the network;
The selecting means includes means for selecting which of the first and second encoders encodes the data and causes the transmitting means to transmit in accordance with the acquired status information. A transmitting device characterized. The transmission device according to claim 1,
Further comprising means for monitoring the load status of the transmitter,
The selection means includes means for selecting which of the first and second encoders encodes the data and transmits the data by the transmission means in accordance with the monitored load situation. A transmitting device characterized. A first encoder that encodes data using a first codec;
A second encoder that encodes the data using a second codec different from the first codec;
Transmitting means capable of transmitting data encoded by the first and second encoders to a receiving apparatus connected via a network;
Means for acquiring status information including at least one of the capabilities and settings of the receiving device from the receiving device via the network;
Selection means for selecting which of the first and second encoders encodes the data to be transmitted by the transmission means in accordance with the acquired status information. A transmitting device. A first encoder that encodes data using a first codec; a second encoder that encodes the data using a second codec different from the first codec; and the first and second The transmission means capable of transmitting the first and second data respectively encoded by the encoder over the network, the monitoring means for monitoring the status of the network, and at least the first data according to the status of the monitored network And a first selection unit that selects whether the data is encoded by one of the second encoders and transmitted by the transmission unit. The first transmission unit transmits the first transmission unit via the network. And means for receiving one of the second data;
A first decoder for decoding the first data using the first codec;
A second decoder for decoding the second data using the second codec;
And second selecting means for selecting which one of the first and second decoders to decode in accordance with the received data of the first and second data. Receiving device. A first encoder that encodes data using a first codec; a second encoder that encodes data using a second codec different from the first codec; and the first and second Transmission means capable of transmitting data encoded by the encoder over the network, monitoring means for monitoring the status of the network, and at least the first and second encoders according to the status of the monitored network A transmission device having first selection means for selecting which encoder to encode the data to be transmitted by the transmission means;
Means for receiving any one of the first and second data transmitted via the network; a first decoder for decoding the first data using the first codec; A second decoder that decodes the second data using a second codec, and one of the first and second decoders depending on the received data of the first and second data And a receiving device having second selecting means for selecting whether to decode the data by a decoder. A method in which a transmitting device connected to a receiving device via a network transmits data,
Monitoring the status of the network;
A first encoder that encodes data using a first codec and a second codec that encodes data using a second codec different from the first codec, depending on at least the condition of the monitored network; Selecting which encoder of the two encoders to encode;
Transmitting the data encoded by the selected encoder to the receiving device. A first encoder that encodes data using a first codec; a second encoder that encodes data using a second codec different from the first codec; and the first and second The transmission means capable of transmitting the first and second data respectively encoded by the encoder over the network, the monitoring means for monitoring the status of the network, and at least the first data according to the status of the monitored network The first and second transmissions are transmitted from the transmission device via the network from a transmission unit having a selection unit that selects which of the encoders encodes the data and transmits the data by the transmission unit. Receiving either one of the data of
And a step of selecting which one of the first and second decoders to decode in accordance with the received data of the first and second data. . A program in which a transmitting device connected to a receiving device via a network transmits data,
In the transmitter,
Monitoring the status of the network;
A first encoder that encodes data using a first codec and a second codec that encodes data using a second codec different from the first codec, depending on at least the condition of the monitored network; Selecting which encoder of the two encoders to encode;
Transmitting the data encoded by the selected encoder to the receiving device. In the receiving device,
A first encoder that encodes data using a first codec; a second encoder that encodes data using a second codec different from the first codec; and the first and second The transmission means capable of transmitting the first and second data respectively encoded by the encoder over the network, the monitoring means for monitoring the status of the network, and at least the first data according to the status of the monitored network The first and second transmissions are transmitted from the transmission device via the network from a transmission unit having a selection unit that selects which of the encoders encodes the data and transmits the data by the transmission unit. Receiving either one of the data of
And a step of selecting which one of the first and second decoders to decode in accordance with the received data of the first and second data. A first encoder that encodes data using a first codec;
A second encoder that encodes the data using a second codec different from the first codec;
Means for acquiring status information including information on at least one of the capabilities and settings of each of a plurality of receiving devices connected via a network from each receiving device via the network;
When it is determined based on the status information that the first and second receiving apparatuses respectively have first and second decoders that decode using the first and second codecs, respectively, And a means for controlling to transmit the first and second data to the first and second receiving devices, respectively. The server device according to claim 14,
Means for determining whether both the first and second encoders are usable, or whether only the first encoder is usable;
The control means is a case where only the first encoder is determined to be usable, and when it is determined that the second receiving device has the first decoder based on the status information, A server apparatus, wherein control is performed so as to transmit one data to the first and second receiving apparatuses. The server device according to claim 14,
Network monitoring means for monitoring the status of the network;
Further comprising means for selecting which of the first and second encoders the data is to be encoded and transmitted by the transmission means according to at least the status of the monitored network. A server device.

Images