JP6543585B2 - Communication control system and method - Google Patents

Description

  The present invention relates to a technology for realizing efficient transfer while maintaining security in communication in which media traffic such as video and audio is transmitted from a large number of devices to devices such as a cloud.

  A large number of technical developments have been made regarding efficient transfer methods of media traffic. One example is a media conferencing system. In order to realize a media conference with multiple participants, it is necessary to make a full mesh connection between each participant, but there may be limitations in bandwidth and terminal processing capability as the number of participants increases. Therefore, media delivery devices such as Multipoint Control Unit (MCU) are typically used for more efficient delivery.

  When using such a media delivery device, it is an important guide in security aspects whether the delivery device can be trusted or not. Conventionally, Real-time Transport Protocol (RTP), which is generally used in IP-based systems, uses the Secure RTP (SRTP) profile that provides security functions while using an MCU at the same time, End-to-End. There was no way to guarantee completeness. For this reason, it is a typical application example that the SRTP key is transferred to a media delivery apparatus such as an MCU, and in this case, it is premised that the MCU or the like can be trusted.

  With regard to the reliability of such MCUs, a framework called Privacy Enhanced RTP Conferencing (PERC) is currently being discussed in the Internet Engineering Task Force (IETF) which is a standardization body, and MCUs, etc. are provided by unreliable third parties. Even if this is the case, extensions are in progress to make security features available. In PERC, it has been proposed to separate an end-to-end authentication area and a Hop-By-Hop authentication area so that the integrity of E2E is secured even if the MCU or the like rewrites the RTP header. As a result, even when the MCU or the like is provided by an unreliable third party, the MCU can be used securely.

  On the other hand, there is a technique called scalable coding as a technique for performing efficient media delivery. The biggest feature of scalable coding is that a single video / audio stream is divided into multiple layers. The layer is designed so that low-resolution video can be generated by decoding only the baseline layer that is the lower layer, and high-resolution video can be generated by decoding to the upper enhancement layer, depending on the bandwidth of the network. By transmitting up to the necessary layers, it is possible to reduce transmission / storage costs and management costs. H.264 / SVC (Scalable Video Coding) is a typical example. In the H.264 / SVC, by combining with the media distribution device, flexible use can be made such that the distribution content is changed according to the distribution destination without changing the processing content of the distribution source.

"RTP: A Transport Protocol for Real-Time Applications" RFC 3550 "The Secure Real-time Transport Protocol (SRTP)" RFC 3711 "A Solution Framework for Private Media in Privacy Enhanced RTP Conferencing" draft-jones-perc-private-media-framework-01 H.264: Advanced video coding for generic audiovisual services "AVC / H.264 scalable extension scheme SVC (Scalable Video Coding) overview", IEICE Technical Report ITS Vol. 109 No. 414 pp 141-145

  Traditional media traffic has often been used between terminals used by humans. However, in the future, cameras and microphones will be installed on many robots and Internet of Things (IoT) devices, such as multi-copters such as drones, reception robots, surveillance robots and cleaning robots, and it is foreseen that a large amount of media streams will be generated. Ru. When these robots / devices show explosive spread, a large amount of traffic may be generated which can not be compared with multimedia traffic assumed to be used by people until now.

  Furthermore, it is predicted that many bottleneck points such as network bandwidth, server processing capacity and storage capacity will occur if the above traffic is transferred and stored in servers such as the cloud via networks such as the Internet. Ru. Therefore, an efficient media transfer method assuming a large number of robots is required.

  On the other hand, from the viewpoint of security, these robots and devices handle video and audio in the sky, in a building, or in a house according to their characteristics, so confidential information of companies, countries, and individuals (information on restricted areas, individuals in a house There is a possibility of handling video and audio containing a lot of information, building internal information, etc., and high security requirements are required. However, it is considered that there are many cases where it is not acceptable to pass an encryption key to a device such as an MCU to make data decodable as in conventional media conferences.

  In the above description, together with the combination of SVC and media distribution device as a specific example for improving the transfer efficiency of media, the approach of PERC in IETF as a specific example for improving security was taken. These two technologies can be combined to solve the problem of achieving both security and transfer efficiency in the robot and IoT device described above. On the other hand, at present, these two approaches can not be combined as they are. That is, layer information of SVC is inserted in the payload of RTP, but in SRTP, the payload is encrypted. For this reason, the media distribution device that does not have the decryption key can not confirm the layer.

  Also, in the conventional media conference, in addition to an arbitrary number of media distribution devices, it is assumed that devices such as transcoders performing media conversion can be connected in any topology, and that media can be transmitted and received bidirectionally. Therefore, while being flexible, it is considered that there is redundancy in improving the efficiency of transfer and communication, and efficient transfer can not be achieved as it is.

  In view of the above-mentioned problems, it is an object of the present invention to provide a technique for realizing efficient media transfer while maintaining security end-to-end.

  In order to solve the above problems, one aspect of the present invention includes a device which is a transmission source of media data, a server which is a transmission destination of the media data, and a transfer node interposed between the device and the server. A system comprising: the media data relates to a system having a media data format configured to place layer information in an unencrypted region of the media data.

  Another aspect of the present invention is a method of transmitting media data from a device to a server, the device transmitting media data to a forwarding node, and the forwarding node using layer information Transferring data to the server, and the media data relates to a method having a media data format configured to place the layer information in an unencrypted area of the media data.

  According to the present invention, efficient media transfer can be realized while maintaining security end-to-end.

FIG. 1 is a diagram showing a system configuration according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of a device according to an embodiment of the present invention. FIG. 3 is a block diagram showing the configuration of a server according to an embodiment of the present invention. FIG. 4 is a block diagram showing the configuration of a forwarding node according to an embodiment of the present invention. FIG. 5 is a block diagram showing the configuration of a conversion node according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a media data format according to an embodiment of the present invention. FIG. 7 is a diagram illustrating a media data format according to another embodiment of the present invention. FIG. 8 is a flowchart showing registration / communication start processing according to an embodiment of the present invention. FIG. 9 is a flow diagram illustrating media transfer change processing according to one embodiment of the present invention. FIG. 10 is a flow diagram illustrating a registration and communication initiation process for a non-compliant device according to one embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described based on the drawings.

  In the embodiments described below, a system is disclosed for achieving efficient media transfer while maintaining security end-to-end.

  First, referring to FIG. 1, a system according to an embodiment of the present invention will be described. FIG. 1 is a diagram showing a system configuration according to an embodiment of the present invention.

  As shown in FIG. 1, the system 10 includes a device 100, a server 200, a forwarding node 300, a translating node 400, and a communication network 500. The device 100, the server 200, the transfer node 300 and the conversion node 400 are communicatively connected via the communication network 500.

  The device 100 is typically realized as various robots or other devices having connectivity with the system 10, and is a device capable of transmitting media data such as video and audio. For example, the device 100 generates a media stream using the equipped camera or microphone, and transmits it to the server 200 via the transfer node 300 or both the transfer node 300 and the conversion node 400 via the communication network 500. Do. The destination server 200 may be one or more. For example, the device 100 is not limited to the following, a sensing device such as a camera, a microphone, or a sensor, a processor for processing media data acquired by the sensing device and controlling the device 100, media data, and the processor The hardware configuration includes a storage device such as a memory for storing a program to be executed, and a communication circuit for communicating media data via the communication network 500.

  The server 200 receives the media stream generated by the device 100 having connectivity with the system 10 via the transfer node 300, and performs processing and accumulation according to the purpose on the received media stream. Furthermore, the server 200 instructs the conversion node 400 and the transfer node 300 to perform processing such as specification of the resolution and the device 100. One server 200 may receive media streams from a plurality of forwarding nodes 300, or information from one forwarding node 300 may be received by a plurality of servers 200. For example, the server 200 is not limited to the following, and is a communication circuit for communicating data via the communication network 500, a processor for processing data, and a storage device such as a memory or storage for storing data. It is realized by the hardware constitution which has etc.

  The forwarding node 300 is realized by a router, a switch or the like, and is an apparatus for relaying communication between the device 100 and the server 200. The forwarding node 300 forwards the media stream received from the device 100 via the communication network 500 to the server 200. The transfer node 300 transfers the media stream not based on the concealed stream information, but based on the checkable header information and the processing instruction and the autonomous judgment transmitted from the server 200.

  The conversion node 400 executes conversion processing for making the device 100 or the server 200 having no connectivity with the system 10 available in the system 10. The conversion node 400 is used between the device 100 and the transfer node 300 or between the server 200 and the transfer node 300 to realize the accommodation of the device 100 having no connectivity with the system 10.

  The configuration of the device 100 according to an embodiment of the present invention will now be described with reference to FIG. FIG. 2 is a block diagram illustrating the configuration of a device 100 according to one embodiment of the present invention.

  As illustrated in FIG. 2, the device 100 includes a sensing unit 110, a media generation unit 120, a security processing unit 130, a storage unit 140, and a communication unit 150.

  The sensing unit 110 acquires external environment information (media stream) related to the external environment of the device 100 such as video and audio via a sensing device such as a camera or a microphone.

  The media generation unit 120 converts the external environment information received from the sensing unit 110 into a media data format to be described later.

  The security processing unit 130 encrypts the media data received from the media generation unit 120 by security processing as described later, and generates an authentication tag.

  The storage unit 140 stores the generated media data in the storage device.

  The communication unit 150 transmits, to the transfer node 300, media data that has been subjected to security processing (such as encryption) received from the security processing unit 130 in accordance with TCP / IP or a communication method compatible with the communication network 500. Further, the communication unit 150 transmits and receives control information necessary to realize the transmission.

  Next, the configuration of the server 200 according to an embodiment of the present invention will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the server 200 according to an embodiment of the present invention.

  As illustrated in FIG. 3, the server 200 includes a communication unit 210, a security processing unit 220, a media processing unit 230, a storage unit 240, a media analysis unit 250, a processing input unit 260, and a processing determination unit 270.

  Communication unit 210 receives media data from transfer node 300 via communication network 500, transfers the received media data to security processing unit 220, and transfers the control command received from processing determination unit 270 to transfer node 300. Do.

  The security processing unit 220 decrypts media data received from the communication unit 210 using security processing described later, and executes authentication processing based on an authentication tag.

  The media processing unit 230 converts media data received from the security processing unit 220 from media data format to be described later into external environment information (media stream) such as video and audio.

  The storage unit 240 stores external environment information such as generated video and audio.

  The media analysis unit 250 analyzes the media stream received from the media processing unit 230, detects an alert such as an emergency situation, a failure, or a failure, and transmits an analysis result to the processing determination unit 270.

  The processing input unit 260 acquires, from the media stream received from the media processing unit 230, the contents of operation input by the provider of the server 200, the user using the application on the server 200, or the operator thereof. As the operation content, without limiting to the following, it is conceivable to switch the video of the specific device 100 to high / low image quality, to stop / start transfer, and the like.

  The processing determination unit 270 determines the processing to be executed based on the results acquired from the media analysis unit 250 and the processing input unit 260, and transmits a control command for instructing the processing to the communication unit 210. The determination criteria include, without limitation to the following, control content (a stream transfer request, stop request, resolution, which gives priority to either the media analysis unit 250 or the processing input unit 260, prioritizes arrival time earlier) It is conceivable to give priority by changing request etc.).

  Next, the configuration of the forwarding node 300 according to an embodiment of the present invention will be described with reference to FIG. FIG. 4 is a block diagram showing the configuration of the forwarding node 300 according to one embodiment of the present invention.

  As illustrated in FIG. 4, the transfer node 300 includes a communication unit 310, a processing determination unit 320, a security processing unit 330, a communication management unit 340, and a communication management database (DB) 350.

  Communication unit 310 transfers media data received from device 100, server 200 and conversion node 400 to communication management unit 340 via communication network 500, and transfers control instructions received from server 200 to processing determination unit 320. .

  The processing determination unit 320 determines transfer of media data according to the control command received from the server 200, and transfers the determination result to the communication management unit 340. The control commands that can be accepted include, but are not limited to, change in resolution (deletion of a specific layer), stop of media data (deletion of all layers), start and end of storage, and the like.

  The security processing unit 330 recalculates the authentication tag of the media data received from the communication management unit 340, and adds the recalculated authentication tag.

  The communication management unit 340 manages the server 200 and the device 100 using the transfer node 300, changes the destination information etc. of the media data transferred from the communication unit 310 to the correct one, and transfers it to the security processing unit 330. .

  The communication management DB 350 manages information of the server 200 and the device 100 using the transfer node 300, and responds to the inquiry from the communication management unit 340.

  Next, with reference to FIG. 5, the configuration of the transform node 400 according to one embodiment of the present invention will be described. FIG. 5 is a block diagram showing a configuration of a transform node 400 according to an embodiment of the present invention.

  As shown in FIG. 5, the conversion node 400 includes a communication unit 410, a media conversion unit 420, a security processing unit 430, and a storage unit 440.

  The communication unit 410 transfers media data received from the device 100 or the server 200 (hereinafter referred to as a non-compliant node) having no connectivity with the system 10 to the media conversion unit 420 and also receives media data received from the media conversion unit 420 Is sent to the forwarding node 300.

  The media conversion unit 420 converts the data of the non-corresponding node received from the communication unit 410 into a media data format described later.

  The security processing unit 430 encrypts the media data received from the media conversion unit 420 by security processing as described later, and generates an authentication tag.

  The storage unit 440 stores the generated media data in the storage device.

  When using the conversion node 400, the security (completeness) of E2E is not secured, but the security between the conversion node 400 and the server 200 is secured.

  Next, media data format according to an embodiment of the present invention will be described with reference to FIGS. FIG. 6 is a diagram illustrating a media data format according to an embodiment of the present invention.

  The media data format shown in FIG. 6 is a data format that enables compatibility between security and transfer efficiency by combining SRTP and SVC based on RTP. In existing RTP-based conferencing systems, Synchronization can be used to indicate which participants are involved in which media stream, even if media streams are mixed through multiple MCUs, transcoders, etc. In addition to Source (SSRC), multiple Contributing Sources (CSRCs) can be described. In the following embodiment, media data is basically transmitted in one direction from the device 100 to the server 200, and it is assumed that there are at most one intermediate transfer node 300. Therefore, there is no need for multiple pieces of CSRC information, and at least one field can be set.

  In RTP, a header specified for each payload is inserted into Payload. For this reason, when using SVC and SRTP, the layer information arranged in the header can not be confirmed. However, in the embodiment shown in FIG. 6, layer information (Layer Type, Layer code) is inserted immediately before the payload (not to be encrypted). Thereby, the transfer node 300 does not look at the contents of the payload section, that is, without acquiring and managing the encryption key from the device 100 or the server 200, the layer in response to a request from the bandwidth or the server 200. It is possible to realize switching of transfer such as change of.

  Also, conventionally, only one bit is assigned to the marker bit (M). However, in the illustrated embodiment, six bits are assigned to the marker bits, and in the use case in the present invention, various types and levels of marking can be made depending on the type of the device 100. In this way, it becomes easy to trace which information is included in which data by exchanging predetermined values between the device 100 and the server 200 depending on the level of urgency and the magnitude of the scale. .

  In the media data format according to the present embodiment, the video and audio communication can be realized by using the profile and payload format used in the conventional RTP as they are except for the portions corresponding to the points deleted above. . Integrity is addressed by diverting the IETF PERC framework. For example, although application of Short EKT (Encrypted Key Transport) or the like can ensure integrity, the present invention is not limited thereto.

  FIG. 7 is a diagram illustrating a media data format according to another embodiment of the present invention. The media data format shown in FIG. 7 is a data format using an existing SRTP profile. In FIG. 6, the header necessary for SVC is inserted while deleting unnecessary header areas in consideration of the topology, but in FIG. 7, the solution is achieved by using the header expansion area possessed by RTP. The X bit indicates the presence or absence of a header extension. When 1 is set to X, a header extension area (header extension) following the CSRC ID group can be used. Layer information (Layer Type, Layer code) similar to that of FIG. 6 is inserted into this header extension area (header extension). By this, even when SRTP is used, the transfer node 300 does not look at the contents of the payload section, that is, without acquiring and managing the encryption key from the device 100 or the server 200, the transfer node 300 In response to the request, it is possible to realize switching of transfer such as layer change.

  In the example of FIG. 6, since the optimization in consideration of the topology is performed, it can be expected that the transfer efficiency is high, but the compatibility with the existing terminal is low. In the example of FIG. 7, while the compatibility with the existing terminal is high, the transfer efficiency is expected to be the same as before.

  Here, the control signal may use an existing protocol (a request response type protocol such as HTTP, SIP, RTSP, etc., or a Publish / Subscribe type protocol such as MQTT), or is defined uniquely. Protocols can also be used. Further, as a data format, an existing format such as SDP may be used, or a format negotiated between the device 100-transfer node 300-server 200 may be used in a format such as JSON.

  The registration and communication initiation process for the device 100 according to one embodiment of the present invention will now be described with reference to FIG. FIG. 8 is a flowchart showing registration / communication start processing according to an embodiment of the present invention.

  As shown in FIG. 8, in step S101, the device 100 transmits a communication start request to the server 200 at a desired communication start timing. Although there is no limitation on the method of specifying the FQDN / URI / IP address of the destination server 200, it is assumed that the hardware / software of the device 100 can be set in advance or dynamically acquired.

  In step S102, when the server 200 desires to use the forwarding node 300, the server 200 notifies the device 100 of the information of the forwarding node 300 and implements redirection.

  In step S103, the device 100 transmits a communication start request to the transfer node 300 based on the notified information on the transfer node 300.

  In step S104, the transfer node 300 transmits a communication start request to the server 200.

  In step S105, the server 200 determines whether transmission is possible based on the information from the transfer node 300, and notifies the transfer node 300 of the determination result. In the present embodiment, the determination result can be transmitted, and the server 200 notifies the transfer node 300 of the permission. When the determination result indicates that transmission is not possible, the server 200 notifies rejection to the forwarding node 300, and ends the processing.

  In step S106, the transfer node 300 determines whether transmission is possible based on the determination result received from the server 200, and notifies the device 100 of the determination result. In this embodiment, the determination result can be transmitted, and the forwarding node 300 notifies the device 100 of the permission.

  In step S107, the device 100 transmits media data in the above media data format to the transfer node 300, and the transfer node 300 transfers the media data to the server 200 according to the layer information. As described above, in the media data received from the device 100, the payload is encrypted for security, but the layer information for improving the transfer efficiency is arranged in the non-encryption area. Therefore, the forwarding node 300 can confirm the layer information without holding the key for decrypting the encryption, and the server 200 can use the media data according to the resolution corresponding to the layer indicated in the layer information. It will be possible to transfer to

  Media transfer change processing according to one embodiment of the present invention will now be described with reference to FIG. FIG. 9 is a flow diagram illustrating media transfer change processing according to one embodiment of the present invention.

  As shown in FIG. 9, in step S201, media data is transmitted from the device 100 to the server 200 via the transfer node 300 after the transmission is permitted by the registration / communication start process described above.

  In step S202, the server 200 determines a transfer policy based on an operator's input, an analysis result of an image, and the like, and transmits a transfer change request to the transfer node 300. The transfer policy may be, for example, switching the video of the device 100 to high / low quality, stopping the transfer, or the like.

  In step S203, the transfer node 300 analyzes the received transfer change request, verifies whether the corresponding operation is valid, and then determines whether the change request can be permitted. If permission is possible, the transfer node 300 notifies the server 200 of change permission.

  In step S204, the forwarding node 300 executes resolution change (deletion / addition of a specific layer) and deletion of media data (deletion of all layers) in accordance with the transmission change request. Thereafter, the transfer node 300 transfers the media data received from the device 100 to the server 200 with the resolution corresponding to the layer after the change.

  Next, with reference to FIG. 10, registration / communication start processing for the non-compliant device 100 according to an embodiment of the present invention will be described. In the example shown in FIG. 8, device 100 was a corresponding device having connectivity with system 10. In the present embodiment, the device 100 is a non-compliant device having no connectivity with the system 10, and in this case, communication between the device 100 and the server 200 is performed via the conversion node 400 together with the transfer node 300. To be done. FIG. 10 is a flow diagram illustrating a registration and communication initiation process for a non-compliant device 100 according to one embodiment of the present invention.

  As shown in FIG. 10, in step S301, the device 100 transmits a communication start request to the conversion node 400. When the non-compliant device 100 connects to the system 10, the translation node 400 may be statically set in the hardware / software of the device 100 or may be dynamically discovered using DNS, UPnP, or the like. Do.

  In step S302, the conversion node 400 converts the received communication start request into a format usable in the system 10, and transmits the converted communication start request to the transfer node 300.

  Steps S303 to S305 are the same as steps S104 to S106 in FIG. 8, and the description thereof will be omitted.

  In step S306, the conversion node 400 converts the received permission into a format that can be received by the non-compliant device 100, and transmits the converted permission to the device 100.

  In step S307, the device 100 transmits media data to the conversion node 400. The conversion node 400 converts the received media data into media data in the above-described media data format by encrypting or the like, and transmits the converted media data to the transfer node 300. The transfer node 300 can confirm layer information arranged in the non-encrypted area of the received media data, and transfer the media data to the server 200 with the resolution corresponding to the layer indicated in the layer information. .

  According to the embodiment described above, it is necessary to transfer all media traffic such as a large amount of video and audio generated and transmitted by a robot such as a drone or other device 100 such as an IoT device to the server 200 etc. in the cloud. It becomes possible to perform efficient transfer by enabling image quality adjustment etc. accordingly. Furthermore, the forwarding node 300 may be a third party different from the provider of the device 100 or the server 200, and can perform efficient forwarding while maintaining end-to-end security. .

  Although the embodiments of the present invention have been described in detail, the present invention is not limited to the specific embodiments described above, and various modifications may be made within the scope of the subject matter of the present invention described in the claims.・ Change is possible.

10 System 100 Device 200 Server 300 Transfer Node 400 Conversion Node

Claims (6)

A device from which media data is sent;
A server to which the media data is sent;
A forwarding node interposed between the device and the server;
A system having
The media data may have a configuration media data format to place the layer information in the non-coding region of the media data,
The forwarding node deletes the layer of the media data in accordance with a control command instructing deletion of a part or all of the layers from the server corresponding to the operation content acquired for the media data. A system for transferring media data from which a selected layer has been deleted to the server .   The transfer node checks layer information disposed in the non-encryption area without decrypting the encryption area of the media data, and transfers the media data to the server using the layer information. The system according to claim 1.   The system according to claim 1, wherein the layer information is disposed immediately before a payload area of the media data format.   The system according to claim 1, wherein the layer information is arranged in a header extension area of the media data format. Further comprising a conversion node interposed between the device and the forwarding node;
The conversion node receives media data in a non-compliant media data format not compatible with the media data format, converts the received media data from the non-compliant media data format to the media data format, and converts the media data the transfer to the transfer node, claims 1 to 4 system set forth in any one. A method of transmitting media data from a device to a server, comprising
The device sending media data to a forwarding node;
The forwarding node forwarding the media data to the server using layer information;
Have
The media data may have a configuration media data format to place the layer information in the non-coding region of the media data,
The forwarding node deletes the layer of the media data in accordance with a control command instructing deletion of a part or all of the layers from the server corresponding to the operation content acquired for the media data. And transferring media data from the deleted layer to the server .

Images