WO2018163418A1 - Communication system, api server used in communication system, headset, and portable communication terminal - Google Patents

Abstract

Provided is a communication system with which it is possible to carry out a comfortable group call even in a low-intensity radio wave environment and an environment having large environmental sound. This communication system 300 is provided with an API server 10 for managing a group call, a portable communication terminal 20 for performing communication via a portable communication network, and a headset 21 for exchanging voice data with the portable communication terminal 20 via short-distance wireless communication. The headset 21 is provided with an utterance enhancement unit for enhancing an utterance part included in voice relative to environmental sound, and the portable communication terminal 20 extracts the utterance part from the voice data received from the headset 21 and transmits the utterance part to a partner in the group call. Communication between the portable communication terminals 20 is controlled by means of a command related to the control of communication quality from the API server 10.

Description

Communication system, API server used in communication system, headset, and portable communication terminal

The present invention relates to a communication system, an API server, a headset, and a mobile communication terminal used in the communication system.

In a communication system for making a many-to-many call within a group using a mobile communication network, the mobile communication terminals that the callers have before each other are registered as a group, and the speaker's voice is encoded. The many-to-many call within the group is performed by exchanging the voice data between the mobile communication terminals registered in the group.

When performing a call between groups using a mobile communication network, voice data encoded with the voice of a speaker is transmitted to a mobile communication terminal of a caller participating in the group via the VoIP server. By transmitting voice data via the VoIP server in this way, the communication load due to many-to-many calls within the group can be reduced to some extent.

Further, when making a call using a mobile communication network, in addition to making a call using a microphone and a speaker provided in the mobile communication terminal, a short distance such as Bluetooth (registered trademark) between the mobile communication terminal and the mobile communication terminal. A headset that is connected and used in a communication system is also used. By using the headset, the mobile communication terminal can pick up the voice of the caller even when the caller is not holding the mobile communication terminal by hand, and the caller can hold the mobile communication terminal by hand. Even when the mobile communication terminal does not hold it, the mobile communication terminal can transmit the conversation sent from the mobile communication terminal of the other party to the caller.

When making many-to-many calls within a group using the above-mentioned conventional technology, if all participants in the call are in a good communication environment, they can talk while exchanging voices with high sound quality and low delay. it can.

However, if you are in an environment with poor radio waves away from base stations such as snowy mountains, sea, construction sites, quarries, airfields, etc., it is also difficult to pick up radio waves that are crowded by many mobile communication terminal users. If the audio is encoded as audio data and then transmitted, the size of the audio data to be transmitted per unit time becomes larger than the communicable bandwidth, resulting in transmission delay of the audio data. There is a problem that it is difficult to continue a comfortable call. At this time, by increasing the compression rate in audio encoding, the audio data to be transmitted per unit time can be appropriately sized with respect to the communicable bandwidth to improve the audio data transmission delay to some extent. In this case, however, the sound quality of the sound obtained when the transmitted sound data is decoded deteriorates, and there is a problem that a good conversation cannot be performed.

In addition, environmental noise such as wind noise, crowd noise, construction sound, mining sound, engine sound, etc. may be a problem when talking on snowy mountains, the sea, in crowds, construction sites, quarries, airfields, etc. is there. When a call is made in such an environment, the microphone used for the call will pick up the ambient sound that is generated in addition to the speaker's utterance, and the voice that mixes the speaker's utterance and the environmental sound will be heard. Data is encoded and transmitted to the mobile communication terminal of the participant of the call, but this environmental sound not only reduces the SN ratio, but also unnecessary audio data of only the environmental sound in which the speaker's utterance does not exist Will cause data delay and the like.

Also, when making a call at a snowy mountain, at sea, in a crowd, at a construction site, a quarry, an airfield, etc., it is often in the middle of an activity other than a call such as driving / operating equipment or playing sports. In such a situation, it is generally necessary to use a transceiver and push the button only during the utterance section to explicitly transmit, but the operation that should be originally performed is hindered by the button operation.

Furthermore, there is also a problem that when the audio data decoded by the mobile communication terminal on the receiving side of the audio data is reproduced, the reproduced audio cannot be heard due to the environmental sound on the receiving side. It is possible to apply noise canceling technology to the environmental sound so that the sound is not heard by the environmental sound, but it is uniform at snowy mountains, at sea, in crowds, construction sites, quarries, airfields, etc. When noise canceling is performed and the environmental sound is cut, there arises a problem that the caller delays the detection of the danger occurring around.

In addition to the above issues, when a group call using a headset and a mobile communication terminal is performed, if a heavy audio encoding method is used, the battery of the headset and the mobile communication terminal is consumed quickly, and the group There is a problem that a call cannot be continued for a long time. In particular, the headset is often a small one that is worn on the ear, and the battery capacity is small compared to the mobile communication terminal. Therefore, the headset and the mobile communication terminal appropriately share the role, It is necessary to encode speech efficiently by combining algorithms with low computational load.

Therefore, an object of the present invention is to provide a communication system capable of performing a comfortable group call even in an environment with a weak radio wave environment or a large environmental sound.

The communication system of the present invention comprises the following three means, and solves the above-mentioned problems that occur in many-to-many communication within a group by linking them together.
Means 1) Means for generating speech data by extracting a human speech portion with high accuracy from speech detected by a headset 2) Dynamic communication quality control means for a weak radio wave environment 3) Considering the environment Noise-resistant regeneration control means

The present invention provides a communication system for performing a group call between a plurality of clients via a VoIP server, comprising an API server for managing the group call, wherein the client communicates via a mobile communication network And a headset for exchanging audio data with the mobile communication terminal by short-range wireless communication, the headset being included in the audio detected by the audio detector and the audio detected by the audio detector A speech emphasizing unit that emphasizes the uttered portion relative to the environmental sound, and voice data received from the mobile communication terminal, and voice detected by the voice detecting unit in the utterance portion of the voice data is compared with ambient noise. And a playback control unit that plays back so that it is relatively easy to hear, and the mobile communication terminal receives from the headset. A noise estimation unit that estimates noise included in the speech data; an utterance candidate determination unit that determines a range that is a candidate for an utterance portion from the speech data based on a result of estimation by the noise estimation unit; and the utterance candidate determination An utterance determination unit that determines a portion that is human speech from a range that is a candidate for an utterance portion of the audio data determined by the unit, and the speech data that is determined to be human speech by the utterance determination unit. A voice data transmission unit that transmits a portion to the VoIP server, and a reproduction voice data transmission unit that transmits the voice data received from the VoIP server to the headset. The API server includes the client, Based on the communication status with the VoIP server, commands related to control of communication quality of the group call are sent to the client and the VoIP. A communication quality control unit for notifying a server, wherein the voice data transmission unit is determined to be human voice by the speech quality determination unit with a communication quality based on a command notified from the communication quality control unit; The audio data portion is encoded and transmitted to the VoIP server.

The present invention reduces the amount of data transferred over a mobile network in a many-to-many group call, thereby reducing the power consumption in portable communication terminals and headsets, and reducing the communication bandwidth. Even when it is not sufficient, it is possible to suppress the voice delay. Furthermore, by automatically detecting only the utterance period, noise is reduced without using hands and does not interfere with other activities, and only the content of the other party's utterance is clearly transmitted, so that the user experience UX (User Experience) ) Can be greatly improved.

1 is a schematic configuration diagram of a communication system according to an embodiment of the present invention. FIG. 2 is a schematic functional block diagram of an API server according to an embodiment of the present invention. 1 is a schematic functional block diagram of a mobile communication terminal according to an embodiment of the present invention. 1 is a schematic functional block diagram of a headset according to an embodiment of the present invention. It is a sequence chart which shows the flow of the process performed on the headset which concerns on the speech detection function by one Embodiment of this invention, and a portable communication terminal. It is a figure which shows the image of the conversion until the audio | voice data transmitted from the audio | voice detected according to the sequence chart of FIG. 5 are produced | generated. It is a sequence chart which shows the flow of the process performed on the headset and portable communication terminal which concern on the audio | voice reproduction | regeneration control function by one Embodiment of this invention. It is a sequence chart which shows the flow of the process performed on the API server, VoIP server, and portable communication terminal which concern on the communication control function when the data transfer delay generate | occur | produces by one Embodiment of this invention. It is a sequence chart which shows the flow of the process performed on the API server, VoIP server, and portable communication terminal which concern on the communication control function when the data transfer condition by one Embodiment of this invention recovers. It is a sequence chart which shows the flow of the process performed on the API server, VoIP server, and portable communication terminal which concern on the communication control function when the communication interruption generate | occur | produces by one Embodiment of this invention. It is a sequence chart which shows the flow of the process performed on the API server, VoIP server, and portable communication terminal which concern on the communication control function when the communication interruption generate | occur | produces by one Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<1. Overall configuration of communication system>
FIG. 1 is a diagram showing a schematic configuration of a communication system according to the present invention. The communication system 300 according to the present invention includes at least one server 1 and the server 1 and GSM (registered trademark), 3G (registered trademark), 4G (registered trademark), WCDMA (registered trademark), and LTE (registered trademark). And a plurality of clients 2 that can be connected via a mobile network.

The server 1 includes at least a VoIP (Voice Over Internet Protocol) server 11 for controlling voice communication between the clients 2, and at least one of the servers 1 included in the communication system 300 includes the client 2. An API (Application Programmable Interface) server 10 that manages connection and allocation of the VoIP server 11 is provided. The server 1 may be configured by a single server computer, or may be configured by preparing a plurality of server computers and implementing the respective functions on the respective server computers. Each server 1 may be distributed and arranged in each region in the world.

The server computer constituting the server 1 includes a storage device (main storage device, auxiliary storage device, etc.) such as a CPU, ROM, RAM, and hard disk, an I / O circuit, and the like. In addition, the server 1 is connected to a wide area network according to a communication standard suitable for wired communication such as TCP / IP, and communicates with the other server 1 through the wide area network. Configured.

The API server 10 exchanges information necessary for the group call with a plurality of clients 2 participating in the group call when a group call is performed in a many-to-many manner, and based on the information obtained there. The VoIP server 11 is instructed to serve as a management server for realizing a group call among a plurality of clients 2 participating in the group call. The API server 10 is mounted on a server computer that constitutes the server 1.

The API server 10 can instruct not only the VoIP server 11 arranged in the same server 1 but also other VoIP servers 11 connectable via the network. This is because the API server 10 specifies the geographical position of the client 2 from information such as the IP addresses of the plurality of clients 2 participating in the group call, and the VoIP server 11 capable of low-latency connection from the client 2 is provided. It is possible to select and distribute the client 2 to the VoIP server 11. In addition, the API server 10 can detect a VoIP server 11 having a low operation rate from a plurality of VoIP servers 11 and distribute the clients 2 to the VoIP server 11.

The VoIP server 11 has a role of controlling voice packet exchange (conversation) between the clients 2 in response to an instruction from the API server 10. The VoIP server 11 is mounted on a server computer constituting the server 1. The VoIP server 11 may be configured as a software switch of a known IP-PBX (Internet Protocol-Private Branch Exchange). The VoIP server 11 has a function of processing voice packets on-memory in order to realize a real-time call between the clients 2.

The client 2 includes a mobile communication terminal 20 provided by a user and a headset 21 connected to the mobile communication terminal 20 by near field communication such as Bluetooth communication.
The mobile communication terminal 20 has a role of performing voice packet communication control in a voice call by a user. The mobile communication terminal 20 can be carried by a user such as a tablet terminal or a smartphone equipped with a storage device (main storage device, auxiliary storage device, etc.) such as a CPU, ROM, RAM, and memory card, and an I / O circuit. Consists of information terminals designed for possible size, shape and weight.

The mobile communication terminal 20 conforms to a communication standard suitable for wireless communication at a long distance, such as GSM (registered trademark), 3G (registered trademark), 4G (registered trademark), WCDMA (registered trademark), LTE (registered trademark). The server 1 and other clients 2 can communicate with each other via a wide area network connected to a base station (not shown).

The mobile communication terminal 20 can communicate audio data with the headset 21 in accordance with a short-range wireless communication standard such as Bluetooth (registered trademark) (hereinafter referred to as “first short-range wireless communication standard”). Composed. In addition, the mobile communication terminal 20 is a short-range wireless communication standard (hereinafter referred to as “second short-range wireless communication”) that can communicate with less power than the first short-range wireless communication standard such as BLE (Bluetooth Low Energy) (registered trademark). It is configured to be able to communicate with the mobile communication terminal 20 at a short distance according to “communication standard”.

The headset 21 creates voice data based on the voice uttered by the user, transmits the created voice data to the mobile communication terminal 20, and plays voice based on the voice data transmitted from the mobile communication terminal 20. Have a role to play. The headset 21 includes a storage device (main storage device and auxiliary storage device) such as a CPU, ROM, RAM, and memory card, and an I / O circuit such as a microphone and a speaker. The headset 21 is configured to be able to communicate audio data with the headset 21 in accordance with a near field communication standard such as Bluetooth (registered trademark). The headset 21 is preferably configured as an open-type headset so that the user who wears it can hear external environmental sounds.

In the communication system 300 according to the present embodiment having the above-described configuration, the VoIP server 11 is installed in each region according to the usage status of the group call service, and the API server 10 centralizes calls by the arranged VoIP server 11. Since management becomes possible, the connection between the clients 2 between the multi-regions can be efficiently operated while recommending communication delay.

<2. Server functional configuration>
FIG. 2 is a diagram showing a schematic functional configuration of the API server 10. The API server 10 includes a call establishment control unit 100, a call quality control unit 110, a client management unit 120, a server management unit 130, and a call group management unit 140. These functional means are realized by the CPU controlling a storage device, an I / O circuit, and the like included in the server computer on which the API server 10 is mounted.

Based on the group call start request from the client 2, the call establishment control unit 100 controls to start a group call between the client 2 and at least one other client 2 included in the group call start request. Is a functional means for performing When the call establishment control unit 100 receives a group call start request from the client 2, if the client 2 that has made the group call start request is not managed by the call group management unit 140, which will be described later, When the call group management unit 140 is instructed to create a call group and the client 2 that has requested group call start is managed by the call group management unit 140, the call group including the client 2 is The call group management unit 140 is instructed to add the client 2 included in the group call start request.

When the call establishment control unit 100 instructs the call group management unit 140 to create a new call group, the call establishment control unit 100 communicates with a plurality of clients 2 participating in the new call group, and determines the geographical position of each client 2. Identify. The call establishment control unit 100 may specify the geographical position of the client 2 based on the IP address of the client 2, or from the position specifying means such as GPS provided in the mobile communication terminal 20 configuring the client 2. The geographical position of the client 2 may be specified based on the information.
When the call establishment control unit 100 identifies the geographical positions of the plurality of clients 2 participating in the new call group, the positions of the plurality of identified clients 2 within the server 1 managed by the server management unit 130 described later. After extracting at least one or more servers 1 arranged in a region where low-latency connection is possible, a server 1 having a VoIP server 11 with a low availability is detected. Then, the call establishment control unit 100 instructs the plurality of clients 2 to start a group call via the VoIP server 11 included in the detected server 1.

The call quality control unit 110 is a functional unit that controls communication quality between a plurality of clients 2 participating in a group call. The call quality control unit 110 monitors the data transfer delay state in the group call by the client 2 managed by the call group management unit 140. When a data transfer delay occurs in a certain client 2, that is, the client 2 The data quality is reduced by suppressing the data quality for the other clients 2 participating in the group call so that the client 2 can maintain the communication when the communication line condition deteriorates. To The call quality control unit 110 may monitor the data transfer delay state of the client 2 by acquiring the communication state of each client 2 from the VoIP server 11 that controls the group call at a predetermined cycle. When the data transfer delay state of the client 2 in which the data transfer delay has occurred is recovered, the call quality control unit 110 cancels the suppression of the data quality for the other clients 2 participating in the group call. Command.

In addition, the communication quality control unit 110 indicates that communication with a client 2 is interrupted when communication with a client 2 is interrupted, that is, when the client 2 becomes unable to communicate due to weak radio waves. To other clients 2 participating in the group call. The call quality control unit 110 may detect that the communication of the client 2 has been interrupted by acquiring the communication status of each client 2 from the VoIP server 11 that controls the group call at a predetermined period. . When the call quality control unit 110 detects that the communication with the client 2 that has lost communication has been recovered, the call quality control unit 110 notifies the other clients 2 participating in the group call to that effect, and the communication is recovered. Then, the client 2 is controlled to rejoin the group call.

The client management unit 120 is a functional unit that manages client information that is information related to the client 2 that makes a group call. The client information managed by the client management unit 120 includes at least identification information for uniquely identifying the client 2 corresponding to the client information, and further information such as the name of the user having the client 2 corresponding to the client information Alternatively, information relating to the geographical position of the client 2 corresponding to the client information may be included. The client management unit 120 receives a client information registration request, a client information request, a client information deletion request, and the like from the client 2 in the same manner as a generally provided service, and performs client information registration, correction, deletion, etc. Processing may be performed.

The server management unit 130 is a functional unit that manages server information that is information related to the server 1 including the VoIP server 11 that can be commanded and controlled from the API server 10. The server information managed by the server management unit 130 includes at least the geographical location of the server and the location (IP address, etc.) of the server on the network, and the operation rate of the VoIP server 11 provided in the server, Information related to the server administrator may be included. The server management unit 130 may perform processing such as registration, correction, and deletion of server information in response to a server information registration operation, a server information correction operation, a server information deletion operation, and the like performed by the administrator of the API server 10. .

The call group management unit 140 is a functional unit that manages call group information, which is information related to a group of clients 2 that are currently making a group call (hereinafter referred to as “client group”). The call group information managed by the call group management unit 140 includes at least information for identifying the client 2 participating in the group call corresponding to the call group information (identification information registered in the client information related to the client 2). The call group information includes information related to the VoIP server used for the group call, and the call group information includes the communication status (data delay status, communication disruption status, etc.) of each client 2 participating in the group call. . The call group management unit 140 receives a call group creation command, a call group deletion command, a call group correction command, etc. from the call establishment control unit 100 and the call quality control unit 110, and creates, corrects, deletes call group information, etc. Processing may be performed.

The API server 10 according to the present embodiment having the above-described configuration is configured to reduce the group call request from each client 2 based on the position of each client 2 participating in the group call and the operating rate of each VoIP server 11 with low delay. The connection can be made to the VoIP server 11 that can be connected. In addition, the API server 10 of the present embodiment detects the alive state of each client 2 that makes a group call via the VoIP server 11 installed in each region, and performs failover processing according to the situation. It is possible to provide an optimal group call service according to the situation without bothering the user.

<3. Client functional configuration>
FIG. 3 is a diagram showing a schematic functional configuration of the mobile communication terminal 20. The mobile communication terminal 20 includes a group call management unit 201, a group call control unit 202, a noise estimation unit 203, an utterance candidate determination unit 204, an utterance determination unit 205, a reproduced audio data transmission unit 207, a communication unit 208, and short-range wireless communication. Part 209. These functional units are realized by the CPU controlling a storage device, an I / O circuit, and the like included in the mobile communication terminal 20.

The group call management unit 201 is a functional unit that exchanges information related to group call management with the API server 10 via the communication unit 208 and manages the start and end of the group call. The group call management unit 201 transmits various requests such as a group call start request, a client addition request, and a group call end request to the API server 10 and performs group call control described later according to a response of the API server 10 to the request. The group call is managed by instructing the unit 202.

The group call control unit 202 transmits / receives audio data to / from other clients 2 participating in the group call based on an instruction from the group call management unit 201 and transmits / receives audio data to / from the headset 21. Is a functional means for controlling. The group call control unit 202 detects the utterance of the voice data related to the user's utterance received from the headset 21 by the noise estimation unit 203, the utterance candidate determination unit 204, and the utterance determination unit 205 described later, and controls the data quality of the voice data. I do.

The noise estimation unit 203 is a functional unit that estimates an average environmental sound from voice data related to a user's utterance received from the headset 21. The voice data concerning the user's utterance received from the headset 21 includes the user's utterance and the environmental sound. As a noise estimation method by the noise estimation unit 203, a least square error (MMSE) estimation is used. Alternatively, a known method such as maximum likelihood method or maximum posterior probability estimation may be used. For example, the noise estimation unit 203 sequentially updates the power spectrum of the environmental sound based on the MMSE standard based on the voice presence probability estimation for each sample frame, and is noise from the voice data using the power spectrum of the environmental sound. The environmental sound may be estimated.

The utterance candidate determination unit 204 is a functional unit that determines a sound different from the average environmental sound from the sound data as an utterance candidate based on the estimation result of the environmental sound that becomes noise by the noise estimation unit 203. The utterance candidate determination unit 204 compares the long-term spectrum fluctuation in units of several frames with the power spectrum of the environmental sound estimated by the noise estimation unit 203, so that the part of the non-stationary audio data is converted into the voice generated by the user's utterance. Judged as data.

The utterance determination unit 205 determines a portion of audio data that is estimated to be an unexpected environmental sound other than a human voice with respect to a portion that the utterance candidate determination unit 204 has determined to be audio data generated by a user's utterance. Is a functional means. The utterance determination unit 205 is uttered from a human throat or the like by performing the estimation of the content ratio of the spectral period component for the portion determined by the utterance candidate determination unit 204 to be voice data by the utterance by the user. It is determined whether the voice data is based on voice. In addition, the utterance determination unit 205 evaluates the distance from the utterer or whether it is a direct wave by estimating the degree of echo from the speech waveform, and whether or not the speech data is based on the speech uttered by the speaker. Determine.

The audio data transmission unit 206 encodes the audio data in a range in which the utterance determination unit 205 determines that it is a sudden environmental sound from the range determined as the utterance candidate by the utterance candidate determination unit 204 and excludes the portion. To the VoIP server. When the audio data transmission unit 206 encodes the audio data, the group call control unit 202 encodes the audio data with the encoding method and communication quality determined based on the command from the communication quality control unit 110 of the API server 10. .

The reproduction voice data transmission unit 207 transmits the decoded voice data received from the VoIP server via the communication unit 208 to the headset 21 via the short-range wireless communication unit 209.

The communication unit 208 is a functional unit that controls communication via the mobile network. The communication unit 208 is realized using a communication interface for a general mobile communication network or the like.
The near field communication unit 209 is a functional unit that controls near field communication such as Bluetooth (registered trademark). The short-range wireless communication unit 209 is realized using a general short-range wireless communication interface.

FIG. 4 is a diagram showing a schematic functional configuration of the headset 21. The headset 21 includes a voice detection unit 211, a speech enhancement unit 212, a playback control unit 213, and a short-range wireless communication unit 216. These functional units are realized by the CPU controlling a storage device, an I / O circuit, and the like included in the headset 21.

The voice detection unit 211 is a functional unit that detects the speech of the user wearing the headset 21 and converts it into voice data. The sound detection unit 211 includes a microphone, an A / D conversion circuit, an encoder for sound data, and the like included in the headset 21. It is desirable that at least two microphones are provided as microphones constituting the sound detection unit 211.

The utterance emphasis unit 212 is a functional means that emphasizes and detects the utterance of the user wearing the headset 21 from the voice data detected and converted by the voice detection unit 211. The speech enhancement unit 212 emphasizes the user's speech relative to the environmental sound using, for example, a known beam forming algorithm. The processing performed by the speech emphasizing unit 212 suppresses the environmental sound included in the audio data relative to the user's speech, thereby improving sound quality and reducing the performance and calculation load of the subsequent signal processing. It becomes possible. The voice data converted by the speech emphasizing unit 212 is transmitted to the mobile communication terminal 20 via the short-range wireless communication unit 216.

The playback control unit 213 is a functional unit that plays back audio data received from the mobile communication terminal 20 via the short-range wireless communication unit 216. The reproduction control unit 213 includes an audio data decoder, a D / A conversion circuit, a speaker, and the like included in the headset 21. The reproduction control unit 213 listens to the user for audio data to be reproduced based on the environmental sound detected by the microphone included in the headset 21 when reproducing the audio in the speech period in the audio data received from the mobile communication terminal 20. Play in a form that is easy to do. The reproduction control unit 213 may perform noise canceling processing based on the ambient noise estimated by the voice detection unit to cancel the environmental sound heard by the user and make it easier to hear the reproduced sound. A process of increasing the playback volume in conjunction with the size of the sound may be performed to make it easier to hear the playback sound.

The client 2 of the present embodiment having the above-described configuration reduces the size of audio data transmitted to the communication path by performing multifaceted audio data processing that links various estimation processes related to speech and environmental sound. However, clear utterance playback is possible. Thereby, labor saving of power consumption in each device constituting the client 2 and a significant improvement in UX (User Experience) can be realized.

Hereinafter, the speech detection function, the communication control function, and the voice reproduction control function, which are characteristic functions of the communication system 300 having the above-described configuration, will be described using a sequence chart showing an operation flow.
<4. Utterance detection function>
FIG. 5 is a sequence chart showing a flow of processing executed on the headset and the mobile communication terminal related to the speech detection function.
[Step SA01] The voice detection unit 211 detects a user's utterance including an environmental sound as a voice and converts it into voice data.
[Step SA02] The utterance emphasizing unit 212 emphasizes the utterance voice of the user included in the voice data converted in step SA01 relative to the environmental sound.
[Step SA03] The short-range wireless communication unit 216 transmits the audio data converted in Step SA02 to the first mobile communication terminal 20.

[Step SA04] The noise estimation unit 203 analyzes the audio data received from the first headset, and estimates the environmental sound that is noise included in the audio data.
[Step SA05] The utterance candidate determination unit 204 determines a sound different from the average environmental sound from the speech data as the utterance candidate based on the estimation result of the environmental sound that becomes noise by the noise estimation unit 203 in Step SA04.
[Step SA06] The utterance determination unit 205 determines the distance of the voice data determined by the utterance candidate determination unit 204 that is the utterance candidate by the user in step SA05 from the sudden environmental sound or the microphone of the headset. The part of the speech data that is estimated to be uttered from a certain position is determined.
[Step SA07] In the group call control unit 202, the utterance determination unit 205 is emitted from a position away from the sudden environmental sound or the headset microphone in step SA06 from the range determined as the utterance candidate in step SA05. The speech data is encoded with the encoding method and the communication quality determined by the exchange with the VoIP server 11 for the speech data in the range excluding the portion determined to be an utterance, and the encoded speech data is converted to the VoIP server Send to.

FIG. 6 is a diagram showing an image of conversion until sound data transmitted from sound detected according to the sequence chart of FIG. 5 is generated. As shown in FIG. 6, in the communication system of the present invention, only the portion necessary for reproducing the utterance is extracted from the detected voice, so that the voice data encoded and transmitted to the VoIP server 11 is usually Compared with audio data transmitted in the communication system, the size can be reduced.

<5. Audio playback control function>
FIG. 7 is a sequence chart showing a flow of processing executed on the headset and the mobile communication terminal related to the audio reproduction control function.
[Step SB01] The group call control unit 202 decodes the data received by the encoding method determined by the exchange with the VoIP server 11 into voice data.
[Step SB02] The reproduction audio data transmission unit 207 transmits the audio data decoded in step SB02 to the second headset 21.

[Step SB03] The sound detection unit 211 detects environmental sound as sound and converts it into sound data.
[Step SB04] The playback control unit 213 performs processing for making it easier to hear the playback sound of the sound data received from the second mobile communication terminal with respect to the environmental sound detected in Step SB03 in the speech section of the sound data. Reproduce.

<6. Communication control function>
FIG. 8 is a sequence chart showing a flow of processing executed on the API server, the VoIP server, and the mobile communication terminal related to the communication control function when a data transfer delay occurs.
[Step SC01] The VoIP server 11 detects the data transfer delay of the second mobile communication terminal 20.
[Step SC02] The VoIP server 11 notifies the API server 10 of the data transfer delay status of the second mobile communication terminal 20.

[Step SC03] The communication quality control unit 110 determines the communication quality according to the data transfer delay state of the second mobile communication terminal 20 notified from the VoIP server 11, and sets the determined communication quality to the VoIP server 11 And the first mobile communication terminal 20 belonging to the same client group as the second mobile communication terminal 20.
[Step SC04] The VoIP server 11 changes the communication quality of the client group to which the second mobile communication terminal 20 belongs to the communication quality commanded in Step SC03.
[Step SC05] The first mobile communication terminal 20 changes the communication quality to the communication quality commanded in Step SC03.

FIG. 9 is a sequence chart showing a flow of processing executed on the API server, the VoIP server, and the mobile communication terminal related to the communication control function when the data transfer state is recovered.
[Step SD01] The VoIP server 11 detects the recovery of the data transfer status of the second mobile communication terminal 20.
[Step SD02] The VoIP server 11 notifies the API server 10 of the recovery of the data transfer status of the second mobile communication terminal 20.

[Step SD03] The communication quality control unit 110 receives the VoIP server 11 and the second mobile phone so as to recover the communication quality in response to the recovery of the data transfer status of the second mobile communication terminal 20 notified from the VoIP server 11. Commands the first mobile communication terminal 20 belonging to the same client group as the communication terminal 20.
[Step SD04] The VoIP server 11 recovers the communication quality of the client group to which the second mobile communication terminal 20 belongs.
[Step SD05] The first mobile communication terminal 20 recovers the communication quality.

FIG. 10 is a sequence chart showing a flow of processing executed on the API server, the VoIP server, and the mobile communication terminal related to the communication control function when communication interruption occurs.
[Step SE01] The VoIP server 11 detects that communication with the second mobile communication terminal 20 has been interrupted.
[Step SE02] The VoIP server 11 notifies the API server 10 of the communication interruption of the second mobile communication terminal 20.

[Step SE03] The communication quality control unit 110 notifies the first mobile communication terminal 20 belonging to the same client group as the second mobile communication terminal 20 that communication with the second mobile communication terminal 20 has been interrupted. .
[Step SE04] The VoIP server 11 changes the information related to the communication state of the second mobile communication terminal 20 to the communication interruption state.
[Step SE05] The first mobile communication terminal 20 changes the information related to the communication state of the second mobile communication terminal 20 to a communication disruption state.

FIG. 11 is a sequence chart showing a flow of processing executed on the API server, the VoIP server, and the mobile communication terminal related to the communication control function when communication interruption occurs.
[Step SF01] The VoIP server 11 detects that the communication status of the second mobile communication terminal 20 has recovered.
[Step SE02] The VoIP server 11 notifies the API server 10 of the recovery of the communication status of the second mobile communication terminal 20.

[Step SE03] The communication quality control unit 110 notifies the first mobile communication terminal 20 belonging to the same client group as the second mobile communication terminal 20 that the communication with the second mobile communication terminal 20 has been recovered. .
[Step SE04] The VoIP server 11 changes the information related to the communication state of the second mobile communication terminal 20 to the normal state.
[Step SE05] The first mobile communication terminal 20 changes the information related to the communication state of the second mobile communication terminal 20 to the normal state.

As mentioned above, although embodiment of this invention was described, this invention is not limited only to the example of embodiment mentioned above, It can implement in a various aspect by adding an appropriate change.

DESCRIPTION OF SYMBOLS 1 Server 2 Client 10 API server 11 VoIP server 20 Portable communication terminal 21 Headset 100 Call establishment control part 110 Call quality control part 120 Client management part 130 Server management part 140 Call group management part 201 Group call management part 202 Group call control part 203 noise estimation unit 204 utterance candidate determination unit 205 utterance determination unit 206 voice data transmission unit 207 reproduction voice data transmission unit 208 communication unit 209 short-range wireless communication unit 211 voice detection unit 212 utterance enhancement unit 213 reproduction control unit 216 short-range wireless Communication unit 300 Communication system

Claims (4)

In a communication system for performing a group call between a plurality of clients via a VoIP server,
An API server for managing the group call;
Each of the plurality of clients includes a mobile communication terminal that communicates via a mobile communication network, and a headset that exchanges voice data with the mobile communication terminal by short-range wireless communication,
The headset is
A voice detection unit for detecting voice;
An utterance emphasizing unit that emphasizes an utterance part included in the voice detected by the voice detecting unit relative to the environmental sound;
A reproduction control unit for reproducing the audio data received from the mobile communication terminal so that the audio detected by the audio detection unit in the utterance portion of the audio data is relatively easy to hear with respect to ambient noise;
With
The mobile communication terminal is
A noise estimator for estimating noise included in audio data received from the headset;
Based on the estimation result by the noise estimation unit, an utterance candidate determination unit that determines a range of utterance part candidates from the voice data;
An utterance determination unit that determines a portion that is human speech from a range that is a candidate for an utterance portion of the voice data determined by the utterance candidate determination unit;
A voice data transmission unit that transmits a portion of the voice data determined to be human voice by the speech determination unit to the VoIP server;
A reproduction audio data transmission unit for transmitting audio data received from the VoIP server to the headset;
With
The API server
A communication quality control unit for notifying each client and the VoIP server of a command related to the control of the communication quality of the group call based on the communication status between the clients and the VoIP server;
The voice data transmitting unit encodes the voice data portion determined to be human voice by the speech determination unit with communication quality based on a command notified from the communication quality control unit, and the VoIP server Send to
A communication system characterized by the above. An API server that manages group calls performed in the communication system according to claim 1,
An API server comprising a communication quality control unit for notifying each client and the VoIP server of a command related to control of the communication quality of the group call based on a communication status between each of the clients and the VoIP server. A headset used in the communication system according to claim 1,
A voice detection unit for detecting voice;
An utterance emphasizing unit that emphasizes an utterance part included in the voice detected by the voice detecting unit relative to the environmental sound;
A reproduction control unit that reproduces audio data received from the mobile communication terminal while noise canceling the audio detected by the audio detection unit in the utterance portion of the audio data;
With headset. A mobile communication terminal used in the communication system according to claim 1,
A noise estimator for estimating noise included in audio data received from the headset;
Based on the estimation result by the noise estimation unit, an utterance candidate determination unit that determines a range of utterance part candidates from the voice data;
An utterance determination unit that determines a portion that is human speech from a range that is a candidate for an utterance portion of the voice data determined by the utterance candidate determination unit;
A voice data transmission unit that transmits a portion of the voice data determined to be human voice by the speech determination unit to the VoIP server;
A reproduction audio data transmission unit for transmitting audio data received from the VoIP server to the headset;
Mobile communication terminal equipped with.

Images