JP2024079861A - Data distribution program and data distribution method - Google Patents

Abstract

To harmonize data transmitted from a plurality of remote areas and distribute the data to a specific space.SOLUTION: A data distribution program causes a computer to serve as a data classification unit, a data adjustment unit, and a data distribution unit. The data classification unit collects transmitted data concerning a sound or an image transmitted from a terminal device of each member authorized to reproduce the sound or the image in a specific space by authentication of an authority code to authorize each member belonging to a predetermined group, and classifies the collected transmitted data into each group. The data adjustment unit adjusts the transmitted data classified into each group, by each group having each member belonging thereto. The data distribution unit distributes the transmitted data adjusted by each group, to the specific space.SELECTED DRAWING: Figure 2

Description

This disclosure relates to a data distribution program and a data distribution method.

Patent Document 1 discloses a video display system that provides video of a live performance to multiple viewers outside the venue, detects the viewers' movements, and displays video corresponding to the detected movements on a display at the venue, giving the viewers the feeling that they are participating in the live performance.

For example, if the video display system detects a specific frequency sound continuously, it determines that the viewer is clapping and displays an image corresponding to the clapping action on the display at the live venue. Also, if the voice recognition means recognizes a specific word, it determines that the viewer is shouting at the audience and displays an image corresponding to the shouting action on the display at the live venue.

JP 2013-021466 A

However, with the conventional technology described in Patent Document 1 above, there is a risk that the multiple images reproduced at the venue will not harmonize due to differences in the transmission method and delay time of the communication network that transmits the data, and the processing power of the devices being used, creating a sense of incongruity. This has led to the problem of how to harmonize data sent from multiple remote locations and different devices and deliver it to a specific space.

Non-limiting examples of the present disclosure contribute to providing a data distribution program and method for harmonizing data transmitted from multiple remote locations or different devices and distributing the data to a specific space.

A data distribution program according to an embodiment of the present disclosure causes a computer to function as a data classification unit that collects transmission data relating to sound or video transmitted from the terminal device of each member who has been granted the authority to reproduce sound or video in a specific space by authenticating an authority code that grants the authority to each member belonging to a specific group, classifies the collected transmission data by group, a data adjustment unit that adjusts the transmission data classified by group for each group to which each member belongs, and a data distribution unit that distributes the transmission data adjusted for each group to the specific space.

In addition, a data distribution method according to one embodiment of the present disclosure includes a computer that performs the steps of collecting transmission data related to sound or video transmitted from the terminal device of each authorized member of a specified group by authenticating an authority code that grants the authority to reproduce sound or video in a specific space to each member of the specified group, classifying the collected transmission data by group, adjusting the transmission data classified by group for each group, and distributing the transmission data adjusted for each group to the specific space.

According to one embodiment of the present disclosure, data sent from multiple remote locations or different devices can be harmonized and delivered to a specific space.

FIG. 1 is a diagram illustrating a configuration example of a live distribution system according to an embodiment of the present disclosure. A diagram showing an example of the main server configuration A diagram showing an example of an edge server configuration 1 is a sequence chart illustrating an example of a process for live distribution. A diagram explaining how to adjust the timing of audio and video Diagram to explain how to suppress feedback and echo Diagram to explain how to suppress feedback and echo Diagram to explain how to suppress feedback and echo

A preferred embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that in this specification and drawings, components having substantially the same functions are designated by the same reference numerals to avoid redundant description.

FIG. 1 is a diagram showing an example configuration of a live streaming system 100 according to an embodiment of the present disclosure. The live streaming system 100 includes a main server 1, multiple edge servers 2, multiple base stations (BSs) 10, multiple terminal devices 20, a camera 51, a speaker 52, and a microphone 53.

The main server 1 is a streaming server that receives information such as audio and video and reproduces it in real time. For example, the main server 1 distributes the audio of people gathering in the EV (EVent) 01 group at an event venue and event scenes to members belonging to the MX (MiXing) 01 group, the GP (GrouP) 01 group, and the BST (BroadcaST) 01 group. The main server 1 and the core network 6 are managed, for example, by a distribution company. The distribution company is, for example, a company that provides the services of the live distribution system 100.

The main server 1 distributes the audio, video, etc., delivered from each group to each group and to viewers watching the event via the Internet 7.

Note that while FIG. 1 shows only one main server 1 and one EV01 group, there may be multiple main servers 1 and multiple event venue groups (EV01 group, EV02 group, ...). Also, the main server 1 may be provided in a cloud within a core network 6 managed by a distributor, as shown in FIG. 1, or in a public cloud 8 outside the core network 6.

The MX01 group is, for example, an orchestra or a choir, and the main server 1 distributes the performances of members belonging to the MX01 group and images of the performances to each group including the EV01 group, viewers, etc.

The GP01 group includes participating members such as cheering squads and fans, and the main server 1 distributes the cheers, cheering songs, and footage of cheering from members of the GP01 group to each group, including the EV01 group, viewers, etc.

The BST01 group may be, for example, commentators belonging to a television station or radio station, or freelance commentators, and the main server 1 distributes live commentary of matches by members belonging to the BST01 group, video footage of the live commentary, etc., to each group including the EV01 group, viewers, etc.

The terminal devices 20 are terminal devices that are owned by members of each group and can connect to 5G or other high-speed communications, such as smartphones, PCs (Personal Computers), PDAs (Personal Digital Assistants), and fixed (stationary) wireless access devices.

When the processor of the terminal device 20 executes the live streaming app stored in the memory, live streaming to the event venue of the EV01 group via the main server 1 is realized. The live streaming app is an app installed on the terminal device 20 that connects to the live streaming system 100 to distribute audio and video data to the event venue and receive audio and video data from the event venue, etc.

The terminal devices 20 linked to the EV01 group (terminal devices 20 of members of the EV01 group such as EV01-01 and EV01-02) include terminal devices such as fixed (stationary) wireless access devices installed at the event venue and connected to a camera 51, speaker 52, and microphone 53. This terminal device 20 outputs sound to the speaker 52 in the EV01 group, and also transmits video and audio data acquired by the camera 51 and microphone 53 to the core network 6.

Furthermore, the terminal devices 20 include terminal devices 20 such as smartphones and PCs that are used to watch events and to make the application required to obtain distributor authority (terminal devices 20 connected to the Internet 7 in FIG. 1). These terminal devices 20 may also be devices connected to the services of other distributors or Internet service providers.

The EV01 group corresponds to one or more facilities or locations where an event is held, such as indoor or outdoor concert venues, stadiums, gymnasiums, studios, community centers, hospitals, and nursing homes.

The edge server 2 is a MEC (Mobile Edge Computing) server that edits and processes data such as audio and video transmitted from the terminal device 20 to the main server 1 at a location close to the terminal device 20. Figure 1 shows multiple edge servers 2 installed in different regions. The data processed by the edge server 2 is transmitted to the main server 1.

The data sent from each edge server 2 is composited (two or more pieces of data edited and processed by the edge servers 2 are timed and bundled together) by the main server 1. By sending audio and video data that has been edited by the distributed edge servers 2 in this way to the main server 1, it is possible to reduce increases in traffic and also the processing load of the main server 1.

For example, the edge server 2 edits and processes data such as the music being played and the audio being played, sent from the terminal device 20 of the MX01 group to the main server 1 at a location close to the terminal device 20.

The edge server 2 also edits and processes data such as cheers, cheering songs, and voices during cheering sent from the terminal devices 20 of the GP01 group to the main server 1 at a location close to the terminal devices 20.

The edge server 2 also edits and processes data such as live commentary and audio during the live broadcast, which are sent from the terminal devices 20 in the BST01 group to the main server 1, at a location close to the terminal devices 20.

Furthermore, the edge server 2 edits and processes data such as audio and video of the event scenery sent from the terminal devices 20 of the EV01 group to the main server 1 at a location close to the terminal devices 20.

In the following explanation, audio and video data that has been edited in the distributed edge servers 2 is sent to the main server 1, and the main server 1 then performs the subsequent processing; however, all processing, including editing, may be performed by the main server 1.

In one embodiment, only the edge servers 2 installed in some regions perform processing such as editing and transmit the edited audio and video data to the main server 1, which then performs subsequent processing, while the edge servers 2 installed in other regions transmit the audio and video data directly to the main server 1, and all processing such as editing is performed by the main server 1.

Members of each group obtain an authorization code for reproducing audio, video, etc. in the EV01 group and a shared key linked to each code, for example through a service (a ticket sales site or a scratch card at a ticket sales store) operated by the company that operates the live distribution system 100 or a company that uses the live distribution system 100 to distribute events, etc.

Members of each group can then broadcast audio, video, and the like by inputting the authorization code and the shared key associated with each code from the live broadcasting app on the terminal device 20. Details of the authorization code and the shared key will be described later.

Camera 51 is an imaging device that captures the scenery at the EV01 group's event venue. Speaker 52 is a sound reproduction device that reproduces the sounds distributed from each group at the EV01 group's event venue. Microphone 53 is a sound collection device that collects sounds at the EV01 group's event venue.

The camera 51, speaker 52, and microphone 53 are connected to the terminal device 20 of the EV01 group, and the image data captured by the camera 51 and the sound data collected by the microphone 53 are transmitted to the core network 6 by the terminal device 20, and the sound data reproduced by the speaker 52 is received from the core network 6.

As shown in FIG. 1, when sound and video of an event are distributed from terminal devices 20 (EV01-01, EV01-02, etc.) linked to the EV01 group, or from terminal devices 20 (not shown) at multiple other related venues, the user can operate the live distribution app on the terminal device 20 to select the terminal device 20 located at the desired viewing position.

The core network 6 is, for example, a 5G core network. The 5G core network is capable of low-latency, high-capacity communication. The 5G core network allows EV01 Group to watch an event in real time from a location away from the event venue, or to remotely participate in an event from a location away from the event venue.

The 5G core network is expected to collect information from millions to hundreds of millions of devices, so a technology called network slicing is attracting attention. Network slicing virtually divides the 5G core network into multiple slices, making it possible, for example, to use one slice for ultra-low latency communications and another slice for high-volume communications.

Even when using a 5G communication core network, if an application server or the like located far away from the terminal device 20 is accessed, the response will be delayed in milliseconds. Therefore, in the 5G core network, edge servers 2 are installed near the terminal device 20, and the processor of each edge server 2 executes various programs recorded in memory, thereby reducing response delays.

The various programs include, for example, mixing programs for conductors and PA (Public Address) engineers (sub-broadcasters of members belonging to the MX01 group), programs to prevent nuisance voices for members belonging to the GP group, and feedback/echo suppression programs.

Figure 2 shows an example of the configuration of the main server 1. The processor of the main server 1 executes programs stored in the memory to realize various functions of the main server 1.

The main server 1 includes a code management and authentication unit 11, a data collection and classification unit 12, a data adjustment unit 13, a data distribution unit 14, and a network transmission and reception unit 15.

The code management and authentication unit 11 is connected to a code issuing system (not shown) that exists in the cloud environment in which the main server 1 in FIG. 1 is located. The code issuing system issues various authorization codes and shared keys for each event, and the data is stored in the code management and authentication unit 11 of the main server 1. The code issuing system may be located outside the cloud environment in which the main server 1 is located.

The code management and authentication unit 11 manages and authenticates the event codes (EV01, EV02, etc.) assigned to each event, the sub-distribution codes (MX01, MX02, etc.), the sub-codes linked to the sub-distribution codes (MX01-01, MX02-01, etc.), the participation codes (GP01-01, GP02-01, etc.), and the commentary codes (BST01, BST02, etc.) in order to reproduce sounds, images, etc. at each event venue. The code management and authentication unit 11 manages, for example, which codes can be used to access which events.

A sub-distribution code such as MX01 is a code that gives a sub-distributor of the MX01 group the authority to adjust and distribute data sent from a terminal device 20 of a user who has an MX01 sub-code (MX01-01, MX01-02, etc.) linked to that sub-distribution code. A sub-distributor is a conductor who uses the live distribution system 100 to live-distribute a performance by an orchestra or a choir or footage of the performance, or a PA engineer who performs mixing work.

For example, to become a sub-broadcaster for MX01, a conductor of an orchestra or the like applies for "sub-broadcaster" authority to the "event organizer" through his or her own live streaming app account. Once the event organizer recognizes the sub-broadcaster as a sub-broadcaster, a sub-distribution code (parent code) such as MX01 linked to the conductor's account is issued, and the sub-broadcaster can obtain the sub-distribution code. At this time, the code management and authentication unit 11 links the sub-distribution code such as MX01 to the conductor's account.

Next, the sub-distributor issues sub-codes (child codes) such as MX01-01, MX01-02, etc. linked to MX01 using their own live distribution app, along with shared keys linked to each sub-code, and distributes these to members of the orchestra or choir. The sub-codes are codes that allow each member of the MX01 group (the group corresponding to the MX01 sub-distribution code) to distribute the EV01 group's performance to the event venue.

The issuance and storage process of sub-codes such as MX01-01 and shared keys is carried out by the code management and authentication unit 11 of the main server 1 via the sub-distributor's live distribution app.

In this way, a sub-broadcaster with a sub-broadcast code of MX01 can adjust the audio and video transmitted from the live streaming app on the terminal device 20 of each member with a sub-code such as MX01-01 by operating the live streaming app on his or her own terminal device 20.

A participation code such as GP01-01 is obtained from online ticket sales or a ticket sales store. A GP01 participation code such as GP01-01 is a code that gives members of the GP01 group the authority to distribute data such as cheers, cheering songs, and audio during cheering.

By purchasing a GP participation code linked to the team or group they support or encourage, supporters and fans can broadcast their cheers at EV01 Group event venues, to other groups participating in EV01 Group events, and to viewers watching the events via the Internet 7.

A commentary code such as BST01 is a code that gives members of the BST01 group the authority to distribute data such as live commentary of the EV01 group's matches and audio during commentary.

The data collection and classification unit 12 collects data such as audio and video transmitted from the terminal devices 20 of the EV01 group and transmits it to the data adjustment unit 13.

The data collection and classification unit 12 also collects data such as audio and video transmitted via BS 10 from the terminal devices 20 of all groups participating in (linked to) the event of the EV01 group (for example, the EV01 group, MX01 group, GP01 group, and BST01 group shown in FIG. 1), classifies the collected data based on the code and composite code linked to each group, and transmits it to the data adjustment unit 13. Here, a composite code is an identification code given to data that combines data from multiple groups.

The data adjustment unit 13 includes a first adjustment unit 13a, a second adjustment unit 13b, a third adjustment unit 13c, a fourth adjustment unit 13d, a fifth adjustment unit 13e, and a sixth adjustment unit 13f that perform various adjustments to the data classified by the data collection and classification unit 12.

The first adjustment unit 13a adjusts the sound and video transmitted from the terminal device 20 of the MX01 group, and transmits the adjusted sound and video data to the sixth adjustment unit 13f. For example, the first adjustment unit 13a grants the sub-distributor the authority to make adjustments to the sound and video data transmitted from the terminal device 20 of the MX01 group. Then, the first adjustment unit 13a accepts adjustment instructions made by the sub-distributor using a live distribution app, and performs mixing processing including volume adjustment, timing adjustment, and video quality adjustment.

The second adjustment unit 13b adjusts the sound and video transmitted from the terminal device 20 of the GP01 group, and transmits the adjusted sound data and video data to the sixth adjustment unit 13f. For example, the second adjustment unit 13b adjusts the volume and sound quality of the sound transmitted from the terminal device 20 of the GP01 group, monitors for nuisance voices (jeers, insults, etc.), suppresses feedback and echoes, adjusts timing, adjusts the image quality of the video, etc., using AI (Artificial Intelligence), and transmits the processed sound data and video data to the sixth adjustment unit 13f.

The third adjustment unit 13c adjusts the sound and video transmitted from the terminal devices 20 in the BST01 group, and transmits the adjusted sound data and video data to the sixth adjustment unit 13f. For example, the third adjustment unit 13c adjusts the volume of the sound and adjusts the image quality of the video transmitted from the terminal devices 20 in the BST01 group using AI or the like, and transmits the adjusted sound data and video data to the sixth adjustment unit 13f.

The fourth adjustment unit 13d uses AI and other techniques to adjust the volume, timing, and image quality of the data transmitted from the terminal devices 20 in the EV01 group. The data that has undergone these processes is then transmitted to the sixth adjustment unit 13f.

The fifth adjustment unit 13e uses AI and other technologies to adjust the volume, timing, image quality, and other aspects of the data that is a combination of data from multiple groups and is sent from the edge server 2. The fifth adjustment unit 13e then sends the data that has been processed in this way to the sixth adjustment unit 13f.

The sixth adjustment unit 13f uses AI and other techniques to combine the data adjusted by the first adjustment unit 13a to the fifth adjustment unit 13e (bundling two or more pieces of data adjusted by the first adjustment unit 13a to the fifth adjustment unit 13e in a timely manner) and to make adjustments such as suppressing feedback and echo, and transmits the adjusted data to the data distribution unit 14 and the network transmission/reception unit 15.

For example, the sixth adjustment unit 13f combines the data from each group, MX01, GP01, BST01, and EV01, obtained from the first adjustment unit 13a to the fifth adjustment unit 13e, to generate data called MBD (Mainserver Bundle) 01, and transmits this MBD01 to the data distribution unit 14 and the network transmission/reception unit 15.

The sixth adjustment unit 13f can also transmit the data acquired from the first adjustment unit 13a to the fifth adjustment unit 13e to the data distribution unit 14 and the network transmission/reception unit 15 without combining the data.

Here, the data MBD01 can include data MBD-MX01 which is a combination of data from the MX01 group, data MBD-GP01 which is a combination of data from the GP01 group, data MBD-BST01 which is a combination of data from the BST01 group, and data MBD-EV01 which is a combination of data from the EV01 group.

The sixth adjustment unit 13f also combines the data from each of the groups MX01, GP01, and BST01 obtained from the first adjustment unit 13a to the third adjustment unit 13c to generate data called MBD02, and transmits this MBD02 to the data distribution unit 14 and the network transmission/reception unit 15.

Furthermore, the sixth adjustment unit 13f generates MBD-MX01 data by combining only the data from the MX01 group acquired from the first adjustment unit 13a, and transmits MBD-MX01 to the distribution unit 14 and the network transmission/reception unit 15. The timing adjustments in each adjustment unit, the methods of suppressing howling and echo, and the combination method will be described later.

Here, the second adjustment unit 13b to the fifth adjustment unit 13e adjust the volume and the like using AI, but the second adjustment unit 13b to the fifth adjustment unit 13e may adjust the volume and the like by receiving instructions from a PA engineer of the distribution company.

The data distribution unit 14 distributes the data acquired from the sixth adjustment unit 13f to terminal devices 20 nationwide. In addition, the audio and video data distributed from the data distribution unit 14 can be muted for each group or composite code combined by the sixth adjustment unit 13f (for example, in the case of MBD01 data, each of the data MBD-MX01, MBD-GP01, MBD-BST01, and MBD-EV01).

The network transceiver unit 15 transmits and receives data between the terminal device 20 connected to the Internet 7 and the public cloud 8.

The processing for other groups such as the MX02, GP02, and BST02 groups is similar to the above processing, so the explanation is omitted.

In addition, here, the data adjustment unit 13 has five adjustment units, the first adjustment unit 13a to the fifth adjustment unit 13e, in addition to the sixth adjustment unit 13f. However, depending on the type of group or composite code corresponding to the data that the main server 1 receives from the edge server 2, adjustment units corresponding to those groups or composite codes are added to the adjustment unit 13 as appropriate.

Figure 3 is a diagram showing an example of the configuration of the edge server 2. The processor of the edge server 2 executes the programs stored in the memory to realize various functions of the edge server 2.

The edge server 2 includes a code management unit 21, a data collection and classification unit 22, a data adjustment unit 23, and a data distribution unit 24.

The code management unit 21 compares the various codes stored and managed in the code management and authentication unit 11 of the main server 1 with the shared key each time a connection is made from the live distribution app described above.

The data collection and classification unit 22 collects in real time via the BS 10 audio and video data transmitted from terminal devices 20 participating in (linked to) an event of the EV01 group for which the code management unit 21 has confirmed that audio and video distribution is taking place, classifies the collected data based on the code or composite code linked to each group, and transmits the data to the data adjustment unit 23.

For example, the data collection and classification unit 22 collects performance sound and video data (e.g., data corresponding to MX01-01, MX01-02, MX02-01, MX02-02, etc.) sent from the terminal devices 20 of members of multiple groups linked to multiple MX sub-distribution codes (e.g., MX01, MX02, etc.), classifies the collected data into each MX group such as MX01, MX02, etc., and transmits it to the data adjustment unit 23.

The data collection and classification unit 22 also collects audio and video data from members who are permitted to distribute using the GP participation code, classifies the collected data into GP groups such as GP01 and GP02, and transmits it to the data adjustment unit 23.

Furthermore, the data collection and classification unit 22 collects data on members who are permitted to distribute by the BST commentary code, such as audio and video data of commentators' live commentary, classifies the collected data into each BST group such as BST01, BST02, etc., and transmits it to the data adjustment unit 23.

The data collection and classification unit 22 also collects audio and video data (MBD01, MBD02, MBD-MX01, etc.) sent from the main server 1 and sends it to the data adjustment unit 23. Note that while FIG. 1 shows a case where there is one main server 1, there may be multiple main servers 1.

The data adjustment unit 23 includes a first adjustment unit 23a, a second adjustment unit 23b, a third adjustment unit 23c, a fourth adjustment unit 23d, a fifth adjustment unit 23e, and a sixth adjustment unit 23f that perform various adjustments to the data classified by the data collection and classification unit 22.

The first adjustment unit 23a opens up specific permissions, such as mixing functions, to sub-distributors of the MX01 group (e.g., the orchestra conductor or PA engineer, etc.).

For example, the first adjustment unit 23a adjusts the volume, adjusts the sound quality, suppresses feedback and echo, adjusts the image quality, adjusts the timing, etc. of data sent from the terminal devices 20 of members belonging to the MX01 group (members who are permitted to distribute their performances by one or more MX01 subcodes). The first adjustment unit 23a transmits the processed sound data and image data to the sixth adjustment unit 23f.

A sub-distributor of the MX01 group who has a sub-distribution code can access the functions of the first adjustment unit 23a from the screen of the live distribution app on his/her own terminal device 20 and perform mixing such as adjusting the volume and timing of data sent from the terminal device 20 of a member (sub-code of MX01) belonging to the MX01 group.

Here, when performers (MX01 subcode) of the MX01 group are participating from all over the country, information on adjustments made by the conductor to the performance of each performer using the live distribution app (e.g., volume adjustment information) is sent to the first adjustment unit 23a of the edge server 2 nationwide that each performer is accessing. Then, based on this information, adjustments such as volume adjustment are made by the first adjustment unit 23a of the edge server 2 nationwide, and the resulting data is sent to the sixth adjustment unit 23f.

The second adjustment unit 23b adjusts the volume, adjusts the timing, suppresses feedback and echo, issues warnings about nuisance voices and images, and temporarily blocks distribution of the audio and video data sent from users in the GP01 group, and transmits the adjusted data to the sixth adjustment unit 23f. Note that the second adjustment unit 23b adjusts the volume, issues warnings about nuisance voices and images, and temporarily blocks distribution using AI, etc.

The third adjustment unit 23c uses AI to make adjustments to the audio and video data sent from users in the BST01 group, such as suppressing feedback and echo, and transmits the adjusted data to the sixth adjustment unit 23f.

The fourth adjustment unit 23d performs processing such as noise cancellation on the data transmitted from the terminal device 20 of the EV01 group using AI or the like. The fourth adjustment unit 23d then transmits the data that has undergone this processing to the sixth adjustment unit 23f.

The fifth adjustment unit 23e uses AI or the like to perform processing such as noise cancellation on data (such as the aforementioned MBD02) that is a combination of data from multiple groups and that is sent from the main server 1. The fifth adjustment unit 23e then sends the data that has been subjected to these processes to the sixth adjustment unit 23f.

The sixth adjustment unit 23f uses AI and other techniques to combine the data adjusted by the first adjustment unit 23a to the fifth adjustment unit 23e (bundling two or more pieces of data adjusted by the first adjustment unit 23a to the fifth adjustment unit 23e in a timely manner) and to make adjustments such as suppressing feedback and echo, and transmits the adjusted data to the data distribution unit 24.

For example, the sixth adjustment unit 23f combines the data of GP01 acquired from the second adjustment unit 23b and MBD02 acquired from the fifth adjustment unit 23e to generate data called EBD (Edge Server Bundle) 02, and transmits the EBD02 to the data distribution unit 24.

The sixth adjustment unit 23f can also transmit the data acquired from the first adjustment unit 23a to the fifth adjustment unit 23e to the data distribution unit 24 as MX01-01, MX01-02, EV01-01, EV01-02, MBD01, or MBD-MX01 without combining the data.

Note that, here, the second adjustment unit 23b to the fifth adjustment unit 23e perform volume adjustment and the like using AI, but the second adjustment unit 23b to the fifth adjustment unit 23e may also perform volume adjustment and the like by receiving instructions from a PA engineer of the distribution company.

The data distribution unit 24 transmits data such as EBD02, MX01-01, EV01-01, MBD01, and MBD-MX01 obtained from the sixth adjustment unit 23f to the main server 1 and the terminal device 20.

In addition, audio and video data such as EBD02 distributed from the data distribution unit 24 can be muted for each combined group or combined code (for example, EBD02 is MBD-MX01, MBD-GP01, MBD-BST01, GP01, etc.).

For example, the data distribution unit 24 transmits data such as MX01-01, MX01-02, EV01-01, and EV01-02 obtained from the sixth adjustment unit 23f to the main server 1.

The data distribution unit 24 also transmits data such as MBD-MX01 acquired from the sixth adjustment unit 23f to the terminal devices 20 of members connected to the BS 10 in the area where the edge server 2 is installed.

Furthermore, the distributor may be able to adjust, combine, and select the destination of each piece of data from the settings screen of the app for the live distribution system 100 provided by the distributor.

The processing for other groups such as the MX02, GP02, and BST02 groups is similar to the above processing, so the explanation is omitted.

In addition, here, the data adjustment unit 23 has five adjustment units, the first adjustment unit 23a to the fifth adjustment unit 23e, in addition to the sixth adjustment unit 23f. However, depending on the type of group or composite code corresponding to the data received by the edge server 2, adjustment units corresponding to those groups or composite codes are appropriately added to the data adjustment unit 23.

Next, the operation of the live distribution system 100 will be described with reference to Figures 4 and 5. The following describes the operation when a cheering song by a brass band (MX01 group) playing in support of Team A, cheers from a cheering squad (GP01 group) supporting Team A, and a live commentary by a commentator (BST01 group) commenting on the game are distributed to a baseball stadium (EV01 group) in an empty baseball game between Team A and Team B.

Figure 4 is a sequence chart that explains an example of processing when members of the MX01 group, GP01 group, and BST01 group linked to event EV01 perform live streaming in sync with video and audio received from the EV01 group. Figure 5 is a diagram that explains a method for adjusting the timing of audio and video.

Here, it is assumed that the terminal devices 20 in the EV01 group, MX01 group, GP01 group, and BST01 group have been authenticated using the various codes and shared keys obtained for the live streaming event as described above.

In addition, in FIG. 4, only one terminal device 20 is shown belonging to each of the EV01 group, the MX01 group, the GP01 group, and the BST01 group, but each group is assumed to have one or more terminal devices 20.

In addition, in FIG. 4, only one edge server 2 is shown that collects audio and video data from terminal devices 20 belonging to each of the MX01 group, the GP01 group, and the BST01 group, but it is assumed that there is one or more such edge servers 2.

As shown in FIG. 4, the terminal device 20 of the EV01 group transmits audio and video data of the event venue of the EV01 group to an edge server 2 located near the event venue (step S1).

Then, the edge server 2 performs various adjustments to the received data as described with reference to FIG. 3 (step S2).

Specifically, the edge server 2 collects sound and video data from the venue transmitted from one or more terminal devices 20 installed at the event venue of the EV01 group. The edge server 2 then performs adjustments such as noise cancellation on the collected data.

The edge server 2 then transmits the data obtained as a result of the various adjustments to the main server 1 (step S3).

The terminal devices 20 of the MX01, GP01, and BST01 groups each transmit audio and video data of the performance, cheers, and commentary to an edge server 2 located near the terminal devices 20 of each group (step S4). Note that it is assumed here that the edge server 2 located near the terminal devices 20 of each group is the same.

The edge server 2 then performs various adjustments to the received data as described with reference to FIG. 3 (step S5).

Specifically, the edge server 2 collects video and audio data transmitted from the terminal devices 20 in the MX01 group, the GP01 group, and the BST01 group.

Then, the edge server 2 classifies the collected data into groups and performs adjustments for each group, such as adjusting the volume of the sound data, adjusting the sound quality, suppressing feedback and echo, adjusting the image quality of the video data, adjusting the timing, and combining the data.

Then, the edge server 2 transmits the data obtained as a result of the various adjustments to the main server 1 (step S6).

Then, the main server 1 collects from the edge server 2 data on which adjustments have been made to the sound and video of the MX01 group, the GP01 group, and the BST01 group, and also collects from the other edge servers 2 data on which adjustments have been made to the sound and video of the MX01 group, the GP01 group, and the BST01 group, and performs the various adjustments described using FIG. 2 on that data (step S7).

Specifically, the main server 1 performs adjustments to the data collected from each edge server 2, such as adjusting the volume of the audio data, adjusting the sound quality, suppressing feedback and echo, adjusting the image quality of the video data, adjusting the timing, and combining the data.

Then, the main server 1 transmits the resulting adjustment data, MBD02, to the edge server 2 that transmitted the sound and video data of the event venue to the main server 1 (step S8).

The edge server 2 that receives the MBD02 data performs various adjustments to the received data as described with reference to FIG. 3 (step S9). The edge server 2 then transmits the adjusted MBD02 data to the terminal device 20 of the EV01 group (step S10). As a result, the sound and video of the performance of the MX01 group, the cheers of the GP01 group, and the commentary of the BST01 group are distributed to the event venue of the EV01 group.

The main server 1 also collects from the edge server 2 data on which adjustments have been made to the sound and video of the EV01 group, and performs adjustments such as volume adjustments and timing adjustments to the sound data as described with reference to FIG. 2, and combines the data on which these adjustments have been made with the data collected in step S7 (step S11).

Then, the main server 1 transmits the resulting adjustment data, MBD01, to the edge server 2 that transmitted the audio and video data of the MX01 group, the GP01 group, and the BST01 group to the main server 1 (step S12).

The edge server 2 that receives the MBD01 data performs various adjustments to the received data as described with reference to FIG. 3 (step S13). The edge server 2 then transmits the adjusted MBD01 data to the terminal devices 20 in the MX01 group, the GP01 group, and the BST01 group (step S14).

As a result, the sound and image of the MX01 group's performance, the cheers from the GP01 group, and the commentary from the BST01 group, as well as the sound and image of the EV01 group's event venue, will be distributed to each member of the MX01 group, GP01 group, and BST01 group.

Here, we will explain how to adjust the timing of sound and video. For example, if each member of the MX01 group performs in a different location, the timing at which the data for each performance reaches the main server 1 may differ depending on the distance from the multiple BSs 10 accessed by each member to the main server 1 and the processing speed of each terminal device 20.

To correct this timing discrepancy, for example, the following process is performed. First, the sub-distributor of the MX01 group operates a live distribution app to transmit silent video that allows a rhythm such as four beats to be understood from the data distribution unit 14 of the main server 1 to the terminal device 20 of each performer of the MX01 group.

At this time, a high-frequency time count sound linked to the rhythm of this video is also added. Alternatively, only a silent video or only a high-frequency time count sound may be transmitted to the terminal device 20 of each performer in the MX01 group.

Here, a high-frequency time count sound is, for example, a sound with a frequency exceeding 16 kHz, which is above the audible range of the average person, and is a sound that allows the user to grasp the position of the sound or video data on the time count, like a video timeline. For example, the time count sound is a sound that indicates the passage of time by changing over time.

By recording and observing this time count sound, it is possible to detect the location where specific sound or video data exists.

In addition, the sub-distributor of the MX01 group may operate a live distribution app to distribute silent video to the terminal device 20 of each performer of the MX01 group, which allows the viewer to understand the musical score and the position of the song other than the rhythm, such as the four beats described above.

For example, in silent video, the color of the displayed notes or lyrics could change in karaoke-like fashion in accordance with the progression of the musical score or song, or specific symbols or pictures could be used to provide cues.

The silent video and high-frequency time count sound described above may be added by the data adjustment unit 13 (first adjustment unit 13a to sixth adjustment unit 13f) of the main server 1 and then transmitted to the terminal device 20, or may be added by the data adjustment unit 23 of the edge server 2 closest to the event venue when venue sound and video are transmitted from the event venue (EV01, EV02 group, etc.) to the main server 1.

Alternatively, a high-frequency time count sound may be played in the event venue, and the microphone 53 of the terminal device 20 may pick up the high-frequency time count sound together with sounds from the venue, and transmit the data.

When each performer in the MX01 group performs while watching the rhythm image linked to the high frequency on the display of the terminal device 20, the high frequency time count sound output from the speaker of the terminal device 20 and the sound and image of the performer are collected and photographed by the terminal device 20, and are transmitted from the terminal device 20 together with the high frequency time count sound via the BS 10 to the edge server 2.

The first adjustment unit 23a of the edge server 2 adjusts the volume of the sound of the performance and the image quality of the video transmitted from each performer's terminal device 20 to an optimal level, and then transmits the adjusted sound and video data, including the high-frequency time count sound, to the main server 1.

The first adjustment unit 13a of the main server 1 receives this data and uses a high-frequency time count sound to adjust the timing of each performer's performance data.

For example, as shown in Figure 5, assume that performance sounds are present on the second and fourth beats of a four-beat sequence of consecutive time count sounds.

In this case, even if the performance data of each performer arrives at the main server 1 at different times, as shown in the diagram on the left, by detecting the time count sound and shifting the time axis of each performance data so that the time count sounds are synchronized, it is possible to realize a synchronized performance by each performer at the event venue, as shown in the diagram on the right.

Note that although a continuous time count sound is added here, a discontinuous time count sound may also be added. In this case as well, by shifting the time axis of each performance data so that the time count sounds are synchronized, it is possible to achieve a performance in which the timing of each performer is synchronized. After that, the data of each performer is combined by the sixth adjustment unit 13f of the main server 1.

In addition, here, the timing of the performances of each performer in the MX01 group is synchronized using a time count sound, but the same process can also be performed for broadcasts from each member of the GP01 group, who cheer in time with the sounds and images of the stadium reproduced by the data, by adding a time count sound to the data of the EV01 group.

In this case, the second adjustment unit 13b of the main server 1 uses a high-frequency time count sound to adjust the timing of the cheering data from each member of the GP01 group, and then the sixth adjustment unit 13f combines the data from each member.

Furthermore, the sixth adjustment unit 13f of the main server 1 uses a high-frequency time count sound to align and combine multiple pieces of data (e.g., data from the MX01 group and data from the GP01 group) whose timing has been adjusted by each adjustment unit.

Also, the sixth adjustment unit 13f may be configured to composite and distribute only data within a predetermined delay time (e.g., 100 msec) based on the timing sound of the data of the EV01 group that arrives at the fourth adjustment unit 13d of the main server 1. The distributor may be able to select this standard or reference value from the settings screen of the app for the live distribution system 100 provided by the distributor.

Then, after the above processing, the sixth adjustment unit 13f erases the high-frequency time count sound. The combined audio and video data is distributed to each group at the event venue, including the EV01 group, and to viewers watching via the Internet 7. The erasure of the high-frequency time count sound may also be performed by the adjustment unit 23 of the edge server 2, the terminal device 20, etc.

In addition, if a sub-distributor listens to a performance distributed from the data distribution unit 14 of the main server 1 and finds that the timing is out of sync between the sound of one instrument and the sound of another instrument, the sub-distributor may use the sub-distributor function of the live distribution app on his/her terminal device 20 to manually adjust the timing (manual timing correction).

In addition, rather than making all of these adjustments manually, the timing can be adjusted first by AI, and then the sub-broadcaster can make further adjustments to the timing.

In addition, information on the timing adjustments made by the sub-distributor to the performance of each performer by operating the live distribution app is transmitted to the first adjustment unit 23a of the edge server 2 nationwide, and the timing adjustment program recorded in the memory of that edge server 2 references the information and executes the timing adjustment process.

As a result, the first adjustment unit 23a of the edge server 2 adjusts the playback timing of the sound whose playback timing is out of sync. The sound data adjusted in this way by the data adjustment unit 23 is sent to the first adjustment unit 13a of the main server 1.

Here, the first adjustment unit 23a of the edge server 2 adjusts the timing, but the first adjustment unit 13a of the main server 1 may adjust the timing. Also, the first adjustment unit 23a of the edge server 2 and the first adjustment unit 13a of the main server 1 may share the timing adjustment process to distribute the load.

In addition, in step S14 shown in FIG. 4, data of MBD01 is transmitted to terminal devices 20 of the MX01 group, the GP01 group, and the BST01 group, but data of MBD01 may also be transmitted to terminal devices 20 or the public cloud 8 via the Internet 7 shown in FIG. 1.

In this case, the data of MBD01 is adjusted for noise processing and the like by the fifth adjustment unit 23e of the edge server 2 located near the terminal device 20, but for data distributed to the Internet 7, no such edge server exists on the transmission path. However, since the line of the Internet 7 is used only for reception, such a function may be included as a function of a dedicated live distribution app (dedicated viewing app) of the terminal device 20 such as a desktop PC.

The same processing as described above is also carried out when a member of the GP01 group performs live streaming, and when a member of the BST01 group performs live streaming.

Next, the method of suppressing feedback and echo will be described in detail with reference to Figs. 5 to 8. Figs. 5 to 8 are diagrams for explaining the method of suppressing feedback and echo implemented by the main server 1 shown in Fig. 2.

Note that, below, we will explain the case where a time count sound 302 is added to sound data 301, but although not specifically mentioned, such sound data 301 is accompanied by video data.

The fourth adjustment unit 13d of the main server 1 shown in FIG. 2 adds a time count sound 302 to the sound data 301 (FIG. 6) distributed from the event venue of the EV01 group, and the data distribution unit 14 transmits the sound data 301 with the time count sound 302 superimposed thereon to the terminal device 20 via the edge server 2.

The time count sound 302 is a sound that indicates the passage of time by changing over time, for example, a sound with a frequency exceeding 16 kHz, which is above the audible range of the average person. By recording and observing the waveform of such a time count sound 302, it is possible to detect the position of specific sound data 301 and remove the detected sound data 301.

When one event is held across multiple venues, high-frequency time count sounds 302 with different frequency bands may be added to multiple terminal devices 20 installed at a first event venue and multiple terminal devices 20 installed at a second event venue.

For example, if time count sounds 302 modulated to include time count information indicating the passage of time (such as time count information corresponding to values 1 to 4 shown in FIG. 5) are used as the multiple time count sounds 302 with different bands, then the time count sounds 302 can be demodulated and the time count information included in the multiple time count sounds 302 can be compared to detect the timing discrepancy between the time count sounds 302 at multiple event venues with different bands. As a result, it is possible to detect the playback timing discrepancy between the sound data 301 collected by the microphone 53 of each terminal device 20 at different event venues, and adjustments can be made to align the playback timing.

In this way, it becomes possible to synchronize the first event venue and the second event venue with the time count sound 302. Also, simply by identifying the high-frequency band contained in the background of the sound received from the participant, it is possible to determine which venue (the first event venue or the second event venue) the participant was watching and cheering at, and it is also possible to easily detect the sound added by the participant and the location of the noise that should be removed. Furthermore, simply by recognizing the high-frequency band contained in the sound data 301 received from each terminal device, it is possible to determine which event venue the sound data 301 should be returned to (sent to), allowing for more fine-grained adjustments.

When the fourth adjustment unit 13d adds the time count sound 302 to the sound data 301 distributed from the event venue of the EV01 group, the fourth adjustment unit 13d stores the sound data 301 distributed from the event venue of the EV01 group in a memory or the like. At that time, the fourth adjustment unit 13d also stores in a memory or the like the time count information of the time count sound 302 within the range in which the sound data 301 exists.

Then, for example, the sound playback unit 303 of the terminal device 20 owned by a member belonging to the GP01 group plays the sounds of the event venue of the EV01 group based on the sound data 301 distributed from the main server 1 via the edge server 2.

In this case, if the sound collection unit 304 of the terminal device 20 collects not only sounds such as cheers from the members but also the sounds from the event venue of the EV01 group that are being played back, and this sound is output from the speaker 52 at the event venue of the EV01 group and collected by the microphone 53 at the event venue of the EV01 group, and this process is repeated, feedback and echoes will occur. Therefore, it is necessary to remove the sounds from the event venue of the EV01 group.

In this case, the data adjustment unit 13 of the main server 1 acquires as sound data 305 a mixture of cheers from members belonging to the GP01 group, sounds from the event venue of the EV01 group being played back, and the time count sound 302.

Then, as shown in FIG. 7, the data adjustment unit 13 observes the time count information of the time count sound 302 contained in the acquired sound data 305, and identifies the period T in which the sound data 301 is included by referring to the time count information of the time count sound 302 stored in a memory or the like.

Then, the data adjustment unit 13 reads out the sound data 301 stored in a memory or the like, and removes the sound data corresponding to the read out sound data 301 from the sound data 305 for the period T, thereby generating sound data 305A including cheers from members belonging to the GP01 group.

In this way, by using the time count sound 302, the position of the sound data 301 distributed from the event venue of the EV01 group can be accurately detected from among sound data that includes cheers and the like, and it can be removed. This makes it possible to suppress the occurrence of howling and echoes, even when the sound from the event venue of the EV01 group reproduced by the sound reproduction unit 303 is collected by the sound collection unit 304.

In addition, by using such a time count sound 302, even if there is a blank period in part of the sound data 305 as shown in FIG. 8, it is possible to detect the position of the sound data 301 distributed from the event venue of the EV01 group based on the existing time count information. As a result, it is possible to accurately remove the sound data 301 from the event venue of the EV01 group.

Note that, although the main server 1 is assumed to add the time count sound 302 and remove the sound data 301 here, these processes may also be performed by edge servers 2 nationwide involved in the EV01 group's events.

It has also been explained that the location of sounds added by participants and noise to be removed can be easily detected. The sounds added by participants and noise to be removed are sound data of a specific frequency band that is not included in the sound data 301 distributed from the event venue (for example, sounds that are unpleasant to other participants, such as metallic sounds).

For example, the data adjustment unit 13 of the main server 1 can detect sound data of a specific frequency band that is not included in the sound data collected at the event venue, and remove the detected sound data from the sound data 305 transmitted from the terminal device 2, thereby removing sounds that are unpleasant to other participants, such as metallic sounds.

The data adjustment unit 13 of the main server 1 also uses AI or the like to detect nuisance voice data that is not included in the sound data collected at the event venue, and removes the detected nuisance voice data from the sound data 305 transmitted from the terminal device 2. This makes it possible to remove nuisance voice that is unpleasant to other participants. Nuisance voice is voice that has been determined to be nuisance voice by a learning model that has been trained in advance to detect nuisance voice. Furthermore, when such nuisance voice is detected by AI, the data adjustment unit 13 may automatically block the voice data transmitted from the terminal device 20 of this participant for a predetermined period of time.

As described above, according to this embodiment, sounds distributed from multiple groups authorized to reproduce sounds at an event venue can be adjusted, and the adjusted sounds can be distributed to the event venue. This allows the sounds reproduced at the event venue to be in harmony even if there are differences in the transmission methods and delay times of the communication networks that transmit the performance and cheering songs, etc., so that even in the case of a match without spectators, for example, it is possible to give the impression that a brass band and spectators are actually present at the venue.

So far, we have explained a method for recreating musical performances, cheers, and the like at an event venue, but this embodiment can also be applied to remote surgery, remote operation of construction equipment, e-sports, and the like. Examples of these are explained below. Note that the same system configuration as described above can be used in the following examples as well.

For example, when performing surgery on a person remotely, the code management and authentication unit 11 of the main server 1 shown in FIG. 2 distributes a sub-code (OP01-01, OP01-02, etc.) linked to a sub-distribution code such as OP (OPeration) 01 to a group (doctor group) consisting of multiple doctors via the edge server 2. Here, the sub-distribution code is obtained, for example, by one of the staff members of the doctor group. The sub-distribution code and the sub-code are collated by the code management and authentication unit 11 and the code management unit 21 of the edge server 2.

The doctor group includes a surgeon and assistants who operate a remote operation device (a type of terminal device 20) to perform surgery. The sub-distribution code and sub-code are authority codes that grant the members of the doctor group the authority to perform remote surgery at the surgical site.

The remote control device is equipped with a handle that accepts operations by the surgeon or assistant, a display unit, etc. The display unit shows images of the operation of the surgical manipulator and the patient's condition, etc.

When a member of the doctor group receives a subcode (OP01-01, OP01-02, etc.) and inputs this subcode into a remote control device, the code management and authentication unit 11 of the main server 1 determines whether the input subcode matches the subcode it manages (authentication of the authorization code).

If these subcodes match, the data collection and classification unit 22 of the edge server 2 shown in FIG. 3 collects from the remote control device an operation command that instructs the surgical manipulator installed at the surgical site to perform an operation corresponding to the operation, and transmits the collected operation command to the data adjustment unit 23.

If the subcode entered by the member does not match the subcode managed by the code management and authentication unit 11, the remote control device is configured not to be able to connect as a doctor group.

The data collection and classification unit 22 of the edge server 2 also collects audio and video data transmitted from a remote control device equipped with a microphone and camera. The data adjustment unit 23 of the edge server 2 then adjusts and combines the operation commands, audio, and video. The resulting data is transmitted to the main server 1.

As mentioned above, "adjusting" refers to the process of adjusting the timing of collected data, eliminating noise, suppressing feedback and echoes, etc., while "combining" refers to bundling multiple sounds, action commands, and video data.

Here, the timing adjustment process may be performed by AI, by staff of the medical group performing the remote surgery, or by AI using a time count sound as described in Figure 5.

Then, the data adjustment unit 13 of the main server 1 that receives this data adjusts and composites the operational commands, sound, and video data received from one or more edge servers 2 again as necessary.

The data is then sent to the surgical manipulator, which operates according to the operation command. In addition, the sound and images captured by the remote control device are played back on speakers and display devices installed at the surgical site.

Meanwhile, the data collection and classification unit 12 of the main server 1 collects data on sounds and images (images capturing the operation of the surgical manipulator, the condition of the patient, etc.) during surgery from multiple terminal devices 20 installed in the operating room, and transmits this data to the data adjustment unit 13.

The data adjustment unit 13 then performs adjustments to the audio and video data during surgery, such as suppressing feedback and echoes, and combines the adjusted audio and video data with audio and video data obtained from the remote control devices of each member of the doctor group via the edge server 2 and adjusted and combined as necessary.

This allows audio and video data showing the scene in the operating room to be timed together with audio and video data showing the conversations between each member of the doctor team.

The resulting data is then sent to the edge server 2, where the data adjustment unit 23 of the edge server 2 performs adjustments such as removing noise and the time count sound, and suppressing feedback and echo. The data is then distributed to the remote control device linked to OP01.

In this way, when a surgeon and assistant who have been given subcodes operate remote control devices in different locations, communication delays can prevent their operation commands from reaching the surgical manipulator at the same time, making it difficult to perform surgery while synchronizing the surgeon and assistant's operations. However, by performing the above-mentioned processing, it becomes possible to perform surgery while synchronizing the surgeon and assistant's operations.

Similarly, the technique of coordinating the timing of dialogue and operations among multiple participants can also be applied to remote control of construction equipment and e-sports.

The above describes the embodiments of the present invention in detail with reference to the drawings, but the functions of each of the above-mentioned devices can be realized by a computer program.

A reading device of a computer that realizes the functions of each of the above-mentioned devices through a program reads the program for realizing the functions of each of the above-mentioned devices from a recording medium on which the program is recorded, and stores the program in a storage device. Alternatively, a network card communicates with a server device connected to the network, and stores the program for realizing the functions of each of the above-mentioned devices downloaded from the server device in a storage device.

The CPU then copies the program stored in the storage device to the RAM, and sequentially reads and executes the instructions contained in the program from the RAM, thereby realizing the functions of each of the above devices.

For example, it is understood that the following aspects also fall within the technical scope of this disclosure:

(1) The data distribution program of the present disclosure causes a computer to function as a data classification unit that collects transmission data related to sound or video transmitted from the terminal device of each member who has been granted the authority to reproduce sound or video in a specific space by authenticating an authority code that grants the authority to each member belonging to a specific group, classifies the collected transmission data by group, a data adjustment unit that adjusts the transmission data classified by group for each group to which each member belongs, and a data distribution unit that distributes the transmission data adjusted for each group to the specific space.

(2) The data adjustment unit detects a deviation in the playback timing of the transmission data transmitted from each member's terminal device based on time count sound data indicating the passage of time transmitted together with the transmission data, and adjusts the playback timing for each group based on the amount of the detected deviation.

(3) When the sound data is included in the transmission data, the data adjustment unit detects the position of the sound data collected in the specific space in the transmission data based on time count sound data indicating the passage of time that is transmitted together with the sound data collected in the specific space, and removes the data collected in the specific space from the transmission data.

(4) The data adjustment unit detects sound or nuisance voice data in a specific frequency band that is not included in the sound data collected in the specific space, and removes the detected sound or nuisance voice data from the transmission data.

(5) The system further includes a sub-code that is linked to a sub-distribution code assigned to each different event that takes place in the specific space and that grants the authority to each member, and a code management and authentication unit that authenticates each member based on a shared key linked to the sub-code.

(6) When the sound data is included in the transmission data transmitted from each member's terminal device, the system further includes a code management and authentication unit that authenticates an owner who has been granted the authority to adjust the volume of the sound data included in the transmission data based on a sub-distribution code issued for each different event that takes place in the specific space, and when the authentication of the owner is successful, the data adjustment unit adjusts the volume of the sound data transmitted from each member's terminal device based on the operation by the owner.

(7) The data adjustment unit adjusts the playback timing between the bundled data of the transmission data adjusted for each group.

(8) The data adjustment unit detects a deviation in the playback timing of the collected data collected in a different specific space based on time count sound data of a different band indicating the passage of time, which is transmitted together with collected data including sound or video data collected in the different specific space, and adjusts the playback timing based on the amount of the detected deviation.

(9) The data distribution method disclosed herein includes a step of collecting transmission data relating to sounds or images transmitted from the terminal devices of each member who has been granted the authority to reproduce sounds or images in a specific space by authenticating an authority code that grants the authority to each member who belongs to a specific group, classifying the collected transmission data by group, adjusting the transmission data classified by group for each group to which each member belongs, and distributing the transmission data adjusted for each group to the specific space.

Specific examples of the present disclosure have been described in detail above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and variations of the specific examples given above.

1 Main server 2 Edge server 6 Core network 7 Internet 8 Public cloud 10 BS
REFERENCE SIGNS LIST 11 Code management section/authentication section 12 Data collection/classification section 13 Data adjustment section 14 Data distribution section 15 Network transmission/reception section 20 Terminal device 21 Code management section 22 Data collection/classification section 23 Data adjustment section 24 Data distribution section 51 Camera 52 Speaker 53 Microphone 100 Live distribution system

Claims (9)

Computer,
a data classification unit that collects transmission data relating to sounds or images transmitted from the terminal devices of each of the members to whom the authority to reproduce sounds or images in a specific space is granted by authentication of an authority code that grants the authority to each member belonging to a predetermined group, and classifies the collected transmission data by group;
a data adjustment unit that adjusts the transmission data classified into groups for each group;
a data distribution unit that distributes the transmission data adjusted for each group to the specific space;
A data distribution program that acts as a The data distribution program according to claim 1, wherein the data adjustment unit detects a deviation in the playback timing of the transmission data transmitted from each member's terminal device based on time count sound data indicating the passage of time transmitted together with the transmission data, and adjusts the playback timing for each group based on the amount of deviation detected. The data distribution program according to claim 1, wherein, when the sound data is included in the transmission data, the data adjustment unit detects the position of the sound data collected in the specific space in the transmission data based on time count sound data indicating the passage of time that is transmitted together with the sound data collected in the specific space, and removes the data collected in the specific space from the transmission data. The data distribution program according to claim 3, wherein the data adjustment unit detects sound or nuisance voice data in a specific frequency band that is not included in the sound data collected in the specific space, and removes the detected sound or nuisance voice data from the transmission data. The data distribution program according to any one of claims 1 to 4, further comprising a sub-code that is linked to a sub-distribution code assigned to each different event that takes place in the specific space and that grants the authority to each member, and a code management and authentication unit that authenticates each member based on a shared key linked to the sub-code. The data distribution program according to any one of claims 1 to 4, further comprising a code management and authentication unit that authenticates an owner who has been granted the authority to adjust the volume of the sound data included in the transmission data based on a sub-distribution code issued for each different event that takes place in the specific space, when the sound data is included in the transmission data transmitted from each member's terminal device, and when the authentication of the owner is successful, the data adjustment unit adjusts the volume of the sound data transmitted from each member's terminal device based on an operation by the owner. The data distribution program according to any one of claims 1 to 6, wherein the data adjustment unit adjusts the playback timing between the bundled data of the transmission data adjusted for each group. The data distribution program according to claim 1, wherein the data adjustment unit detects a deviation in the playback timing of the collected data collected in a different specific space based on time count sound data of a different band that indicates the passage of time and is transmitted together with collected data including sound or video data collected in the different specific space, and adjusts the playback timing based on the amount of the detected deviation. The computer
a step of collecting transmission data relating to sounds or images transmitted from terminal devices of each of the members to whom the authority has been granted by authenticating an authority code that grants the authority to reproduce sounds or images in a specific space to each member belonging to a predetermined group, and classifying the collected transmission data by group;
adjusting the transmission data classified into groups for each group;
Distributing the transmission data adjusted for each group to the specific space;
A data delivery method that performs:

Images