JP2025037354A - Communication support device and communication support method - Google Patents

Abstract

[Problem] When conducting a two-way call, telephone conference, or online conference, each speaker (participant) is appropriately supported, improving the convenience for all speakers and improving the quality of the call or conference. [Solution] A voice recognition unit 1211 converts voice information from each speaker into text data. An ambiguous utterance detection unit 1212 analyzes the text data from the voice recognition unit 1211 and detects a specific portion that requires support. A keyword extraction unit 1213, a search execution unit 123, and a correction sentence creation unit 125 function to create a message corresponding to the specific portion, and a correction provision unit 126 converts the message into text information or voice information and provides it to the speaker. [Selected Figure] Figure 2

Description

This invention relates to a device and method for supporting each speaker when multiple speakers make a call to send and receive voice information over a network.

Patent Document 1, which will be described later, discloses an invention relating to a sales support system. This sales support system transmits the conversation between the customer and the salesperson as voice data from a communication terminal device via a network to a sales support server, where any errors in what was said are corrected in real time and transmitted to the communication terminal device as an error correction message. This allows any errors in what was said to be corrected immediately. This avoids problems that may occur when negotiations proceed with incorrect information being conveyed, and prevents inconvenience to the customer.

JP 2002-288352 A

The sales support system disclosed in the above-mentioned Patent Document 1 is intended to support salesmen who deal with customers, and is not intended to correct customer mistakes. In recent years, telephone conferences and online conferences (Web conferences) have become commonplace. In a telephone conference, multiple participants located in remote locations send and receive voice messages to each other through telephone terminals connected via telephone lines, and hold a conference in real time. In an online conference (Web conference), participants in remote locations are connected using the Internet environment and devices such as a PC (Personal Computer) or a mobile communication terminal such as a smartphone, and audio and video information is sent and received between the participants, and a conference is held in real time.

In the case of telephone conferences and web conferences, participants are not limited to two people, the caller and the callee, as in conventional telephone communication, but can be two or more people. For this reason, for example, simply detecting an error in the content of a call for only one person, the organizer of the conference, and notifying only that organizer will not improve the convenience for all participants, and will not lead to an improvement in the quality of the conference as a whole. In addition to correcting errors in the content of the call, it is also necessary to supplement with accurate information any unclear matters that were discussed in the conference, such as names or events. Furthermore, it is also necessary to prevent the transmission of inappropriate remarks from the perspective of compliance violations and harassment, taking into account the participants and content of the conference.

In view of the above, the object of the present invention is to provide appropriate support to each speaker (participant) in, for example, a two-party call, telephone conference, or online conference, thereby improving the convenience for all speakers and improving the quality of the two-party call, telephone conference, or online conference.

In order to solve the above problem, the communication support device of the present invention described in claim 1 comprises:
A communication support device that relays voice information from each speaker when a call is made between a plurality of speakers by transmitting and receiving voice information over a network, comprising:
a speech recognition means for converting speech information from each speaker into text data;
a detection means for analyzing the text data from the speech recognition means and detecting a predetermined portion requiring assistance;
a creating means for creating a message corresponding to the predetermined portion when the detecting means detects the predetermined portion, or for creating processed voice information by processing a portion of the voice information corresponding to the predetermined portion;
and a providing means for converting the message into text information or into audio information and providing the converted audio information to a speaker who provided the audio information from which at least the specified portion was detected, or for providing the processed audio information to a speaker other than the speaker who provided the audio information from which the specified portion was detected.

According to the communication support device of the invention described in claim 1, the communication support device relays voice information from each speaker when a call is made between multiple speakers in which voice information is transmitted and received over a network. In the communication support device, the voice information from each speaker is converted into text data by the voice recognition means. The detection means analyzes the text data from the voice recognition means and detects a specific portion that requires assistance.

When the detection means detects the specified portion, the creation means creates a message corresponding to the specified portion, or processes the portion of the voice information from the speaker that corresponds to the specified portion to create processed voice information. The provision means converts the message created by the creation means into text data or voice information and provides it to at least the speaker who provided the voice information that detected the specified portion, or provides the processed voice information to a speaker other than the speaker who provided the voice information that detected the specified portion.

According to this invention, when a two-party call, telephone conference, or online conference is held, it is possible to appropriately support each speaker (participant). This improves the convenience for all speakers and also improves the quality of the two-party call, telephone conference, or online conference.

1 is a diagram for explaining a configuration example of a communication support system according to an embodiment; 1 is a block diagram for explaining a configuration example of a telephone control device to which a first embodiment of a communication support device according to the present invention is applied; 4 is a sequence diagram for explaining a process in a call support system configured using the telephone control device of the first embodiment. FIG. FIG. 11 is a block diagram for explaining a configuration example of a telephone control device to which a second embodiment of a communication support device according to the present invention is applied. FIG. 11 is a sequence diagram for explaining a process in a call support system configured using a telephone control device according to a second embodiment. FIG. 13 is a block diagram for explaining a configuration example of a telephone control device to which a third embodiment of a communication support device according to the present invention is applied. FIG. 13 is a sequence diagram for explaining a process in a call support system configured using a telephone control device according to a third embodiment.

Below, an embodiment of the device and method according to the present invention will be described with reference to the drawings. The device and method according to the present invention can be applied to, for example, telephone control devices such as a main unit that controls telephone calls, a PBX (Private Branch eXchange), and a SIP (Session Initiation Protocol) server, a conference server for online conferences, and a cloud PBX. Here, telephone control devices such as a main unit, a PBX, and a SIP server are devices installed on the premises of a company or the like to build a so-called business phone system, and relay the voices of both callers.

The conference server is a server device installed on the Internet to realize online conferences, and relays the voices of all participants in the online conference. The cloud PBX builds PBX functions on the Internet (on the cloud), allowing business phone functions to be used over an Internet line, and relays the voices of both callers. A telephone conference can be realized by providing a telephone control device or cloud PBX with the functionality to become an access point for the telephone conference.

The device and method of the present invention are applicable to various devices that relay the voice information of all speakers who make a call or hold a conference, such as a telephone control device, a conference server, and a cloud PBX. In other words, the device and method of the present invention are applicable to devices that relay voice information from each speaker when a call is made between multiple speakers in which voice information is sent and received over various networks. In the embodiment described below, for simplicity, the device and method of the present invention are applied to a telephone control device installed on the premises of an office or the like to build a so-called business phone system, and a call between two people is described as an example.

In the narrow sense, the term "call" means talking on the telephone. However, in this specification, the term "call" includes conversation between multiple speakers by sending and receiving voice information in real time over a specified network. In other words, in this specification, the term "call" does not only mean telephone communication between two parties over a telephone line, but also includes cases where multiple speakers hold meetings or conferences by voice in real time, such as telephone conferences and online meetings.

[Example of configuration of a call support system]
Fig. 1 is a diagram for explaining an example of the configuration of a telephone communication support system according to an embodiment. In Fig. 1, a wide area network 6 shown in the center includes an external telephone network 61 and an IP (Internet Protocol) network 62. The external telephone network 61 includes a public switched telephone network, a mobile phone network, etc., and mainly realizes voice communication services. The IP network 62 refers to a computer network interconnected using Internet Protocol Suite technology, and is equivalent to the so-called "Internet."

Plural internal telephone terminals (hereafter referred to as telephone terminals) 3(1), 3(2), 3(3), ... are connected to telephone control devices 1A, 1B, 1C, which are connected to an external telephone network 61 and an IP network 62, via an internal telephone network 2, forming a business phone system. The telephone control devices 1A, 1B, 1C control connections between internal and external lines and connections within internal lines. In other words, the telephone control devices 1A, 1B, 1C are connected to multiple telephone terminals 3(1), 3(2), 3(3), ..., and perform so-called call control such as connecting, disconnecting, and transferring communications between internal and external lines and within internal lines.

The telephone control devices 1A, 1B, and 1C are also connected to the IP network 62 because it is possible to use IP telephone services that use VoIP technology through the IP network 62. The telephone control devices are divided into 1A, 1B, and 1C because, in addition to the function of performing call control, each device has different functions. The different functions of the telephone control devices 1A, 1B, and 1C will be described in detail later.

The telephone control devices 1A, 1B, and 1C are also connected to an internal information server 5(1) via a LAN (Local Area Network) 4. The internal information server 5(1) is a server device that manages various information used within the company, such as a customer information DB (Data Base), a transaction information DB, and a product information DB. The internal information server 5(1) is not limited to one unit, and multiple internal information servers 5(1), 5(2), ... may be provided, each managing different information. In this way, the telephone control devices 1A, 1B, and 1C are connected to the internal information server 5(1) etc. via the LAN 4, making it possible to obtain the necessary information from the internal information server 5(1) etc.

As shown in FIG. 1, an outside telephone terminal 7(1) and an information providing server 8(1) are connected to the wide area network 6. Although not shown in FIG. 1, a mobile communication terminal such as a smartphone can also be connected via a base station of a mobile phone network. Although only one outside telephone terminal 7(1) is shown in FIG. 1, in reality, many outside telephone terminals such as outside telephone terminals 7(2), 7(3), etc. are connected. The outside telephone terminals 7(1), 7(2), 7(3), etc. may be connected to the wide area network 6 like a fixed telephone terminal located in a home, or may be connected via another telephone control device and used as part of a business phone system.

As shown in FIG. 1, an information providing server 8(1) is connected to the wide area network 6. Although only one information providing server 8(1) is shown in FIG. 1, in reality, many information providing servers are connected, such as information providing servers 8(2), 8(3), .... Information providing servers 8(1) and the like are server devices that provide various types of information, such as train transfer information, past and current weather information (history) and future weather forecasts, and past news information.

The telephone control devices 1A, 1B, and 1C of the call support system shown in FIG. 1 do not just control the connection of call lines between internal and external lines and the connection of call lines within an internal line. The telephone control devices 1A, 1B, and 1C of this embodiment analyze the call voices from both speakers and have new functions such as (1) post-correction (correction of ambiguous utterances), (2) advance supplementation (supplementation of unclear utterances), and (3) advance prevention (overwriting of inappropriate utterances). Of course, it is also possible for one telephone control device to have all three functions shown as (1) to (3).

However, in the following, for simplicity, telephone control devices 1A, 1B, and 1C will be described as each having a different function among the above (1) to (3). That is, telephone control device 1A of the first embodiment has a (1) post-correction function (correction of ambiguous remarks). Telephone control device 1B of the second embodiment has a (2) advance supplementation function (supplementation of unclear remarks). Telephone control device 1C of the third embodiment has a (3) advance prevention function (overwriting inappropriate remarks).

In addition, the telephone terminals 3(1), 3(2), 3(3), ... shown in FIG. 1 have the same basic configuration, and therefore in the following explanation, unless otherwise specified, telephone terminals 3(1), 3(2), 3(3), ... will be collectively referred to as telephone terminal 3. Similarly, the external telephone terminals 7(1), 7(2), 7(3), ... also have the same basic configuration, and therefore in the following explanation, unless otherwise specified, telephone terminals 7(1), 7(2), 7(3), ... will be collectively referred to as external telephone terminal 7.

As described above, in the call support system shown in FIG. 1, calls between telephone terminals and external telephone terminals 7 (calls between internal and external lines) and calls between telephone terminals 3 (calls within an internal line) are possible. However, for simplicity of explanation, the following will be described using an example in which a call line is connected between telephone terminals 3 and external telephone terminals 7 and a call is made (calls between internal and external lines). Below, the telephone control devices 1A, 1B, and 1C of the first, second, and third embodiments will be described in detail.

[First embodiment]
<Configuration Example of Telephone Control Device 1A of First Embodiment>
The telephone control device 1A of the first embodiment has a function of analyzing the voices of both speakers who are connected to a telephone line and making post-correction (correction of ambiguous utterances). Fig. 2 is a block diagram for explaining a configuration example of the telephone control device 1A to which the first embodiment of the communication support device according to the present invention is applied.

In FIG. 2, connection end 101T constitutes the connection end to the external telephone network 61, and telephone network I/F (Interface) 101 is a part that performs communication processing through the external telephone network 61. That is, telephone network I/F 101 converts signals sent to the device itself via the external telephone network 61 into signals in a format that can be processed by the device and imports them. In addition, telephone network I/F 101 converts signals to be sent from the device to the intended recipient into signals in a transmission format and sends them to the external telephone network 61 to transmit them to the recipient.

The control unit 102 is a microprocessor equipped with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), non-volatile memory, etc. (not shown), and controls each part of the telephone control device 1A. The storage device 103 is a device unit consisting of a recording medium such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and its driver, and records, reads, modifies, and deletes various data on the recording medium. In addition to storing and holding necessary data and programs, the storage device 103 is also used as a working area for temporarily storing intermediate data generated in various processes.

The terminal information file 104 is created in a recording device such as an HDD or SSD, and stores and holds various information about the telephone terminal 3, including the extension number of the telephone terminal 3 connected (housed) to the device, as well as information indicating the current status of the telephone terminal 3. Call control for the telephone terminal 3 is performed using the information stored in this terminal information file 104.

The ambiguous utterance dictionary 105A is created in a recording device such as a HDD or SSD, and holds various dictionary data for detecting ambiguous utterances in the voices of both speakers who are talking over a connected telephone line. Examples of the dictionary data include ambiguous expressions such as "probably," "I think," "if I remember correctly," and "probably," as well as expressions that include numbers such as "xx minutes," "xx hours," and "xx yen." Expressions that include numbers are included because the numerical portion may not necessarily be correct.

The search destination determination dictionary 106 is also created in a recording device such as an HDD or SSD. The search destination determination dictionary 106 holds dictionary data for determining whether to search for stored information such as the internal information server 5 (1), ... or the information providing server 8 (1), ... on the IP network 62 in order to search for the exact content of an ambiguous statement. The dictionary data is simply a set of search keywords, such as "search keyword: internal" and "search keyword: external", paired with information indicating whether the search destination is internal or external.

Examples of dictionary data in the search destination determination dictionary 106 include "Product number 1234: inside", "Sales representative: inside", "From XX station to □□ station: outside", and "Weather on XX month, △△ day: outside". Here, "inside" refers to the internal information server 5(1), ... connected to the LAN 4, and "outside" refers to the information providing server 8(1), ... on the IP network 62. By narrowing down the search destination using the search destination determination dictionary 106, it is possible to appropriately and quickly search for the exact content of an ambiguous statement.

The connection end 107T constitutes a connection end to the internal telephone network 2. The internal I/F (Interface) 107 enables communication between the telephone control device 1A and each of the telephone terminals 3 accommodated in the telephone control device 1A through the internal telephone network 2. Therefore, signals from the telephone terminals 3 are converted by the connection I/F 107 into signals in a format that can be processed by the device itself and then imported. Furthermore, signals from the telephone control device 1A to the telephone terminals 3 are converted by the connection I/F 107 into signals in a format for transmission and then transmitted to the telephone terminals 3. Therefore, communication through the internal telephone network 2 is carried out through the connection end 107T and the connection I/F 107.

The connection end 108T constitutes the connection end to the LAN4. The LAN I/F (Interface) 108 enables communication between the telephone control device 1A and the internal information server 5 (1), ... via the LAN4. Therefore, signals from the telephone control device 1A to the internal information server 5 (1), ... are converted by the LAN I/F 108 into signals in a format for transmission and transmitted to the internal information server 5 (1), .... Furthermore, signals from the internal information server 5 (1), ... are converted by the LAN I/F 108 into signals in a format that can be processed by the device itself and then imported. Therefore, communication through the LAN4 is carried out through the connection end 108T and the LAN I/F 108.

Under the control of the control unit 102, the call control unit 109 uses the management information in the terminal information file 104 to perform call control such as making and receiving calls, answering and disconnecting calls of the telephone terminal 3. As shown in FIG. 2, the call control unit 109 is equipped with an outgoing call control unit 109S and an incoming call control unit 109R. When the call control unit 109 receives an outgoing call (outgoing call request) from a subordinate telephone terminal 3, the outgoing call control unit 109S functions to call the specified destination, and when the destination responds, the telephone line is connected to enable the call.

When the call control unit 109 receives an incoming call addressed to itself (a call notification from the other party), the incoming call control unit 109R functions to notify the subordinate telephone terminal 3 of the incoming call. This causes the telephone terminal 3 to emit a ring tone from the sound emission unit (ringer), notifying the occurrence of the incoming call. When an operation to answer the incoming call (off-hook operation) is performed at any of the telephone terminals 3, the incoming call control unit 109R detects this, answers the incoming call, and connects a call line to the off-hook telephone terminal 3 to enable the call.

After this, the processes of putting the connected telephone line on hold, transferring, and releasing are processed under the control of the control unit 102 in response to requests from the subordinate telephone terminals 3. The control unit 102 also controls the turning on and off of LEDs (Light Emitting Diodes) equipped in the subordinate telephone terminals 3 and controls display on the display.

The connection end 110T constitutes the connection end to the IP network 62. The communication I/F (Interface) 110 is a part that performs communication processing through the IP network 62. As a result, the telephone control device 1A can access the information providing server 8(1), ..., etc. on the IP network 62 through the communication I/F 110 and the connection end 110T, and can receive the necessary information by performing a search.

The conversation support processing unit 120A is the part that realizes the function of analyzing the voices of both speakers who are connected via telephone lines and performing post-correction (correction of ambiguous utterances). As shown in FIG. 2, the conversation support processing unit 120A has a voice processing unit 121A consisting of a voice recognition unit 1211, an ambiguous utterance detection unit 1212, and a keyword extraction unit 1213. Furthermore, the conversation support processing unit 120A has a search destination determination unit 122, a search execution unit 123, a correctness determination unit 124, a correction sentence creation unit 125, and a correction providing unit 126.

The voice recognition unit 1211 connects a telephone line and converts the voices of both speakers into text data. In this case, the voice recognition unit 1211 converts the voices of the telephone call from the telephone terminal 3 and the external telephone terminal 7 into text data in such a way that they can be distinguished from each other. The ambiguous utterance detection unit 1212 analyzes the text data converted from the voices of the speakers by dividing them into phrases and words, and detects ambiguous utterances by referring to the ambiguous utterance dictionary 105A.

The keyword extraction unit 1213 extracts search keywords from the text data converted by the speech recognition unit 1211 to search for the exact content of the ambiguous utterance portion detected by the ambiguous utterance detection unit 1212. The search destination determination unit 122 refers to the search destination determination dictionary 106 using the search keywords extracted by the keyword extraction unit 1213, and determines whether to search the internal information server 5(1), ... or the information providing server 8(1), ... on the IP network 62.

The search execution unit 123 performs processing to execute a search using the search keywords extracted by the keyword extraction unit 1213, with the information stored in the search destination determined by the search destination determination unit 122 as the search target. In this case, if the search destination is the internal information server 5(1), ..., a specified search program is executed to obtain information (search results) that matches the search keywords from among the information stored in the internal information server 5(1), .... Also, if the search destination is an information providing server 8(1), ... on the IP network 62, the search execution unit 123 executes a specified browser (web page viewing software), executes a search using the search keywords, and obtains information (search results) that matches the search keywords.

The correctness determination unit 124 compares the content of the ambiguous utterance part detected by the ambiguous utterance detection unit 1212 with the search results obtained by the search execution unit 123, and performs a correctness determination of whether the content of the ambiguous utterance part is correct or incorrect. If the correctness determination unit 124 determines that the content of the ambiguous utterance part detected by the ambiguous utterance detection unit 1212 is correct, the processing for that ambiguous utterance part ends. However, if the correctness determination unit 124 determines that the content of the ambiguous utterance part detected by the ambiguous utterance detection unit 1212 is incorrect, the correction sentence creation unit 125 functions.

The correction sentence creation unit 125 creates a correction sentence to correct the content of the detected ambiguous utterance portion, taking into consideration the search results obtained from the search execution unit 123 and information such as search keywords. The correction providing unit 126 converts the correction sentence (text data) created by the correction sentence creation unit 125 into voice information and provides it to the speaker connected to the telephone line via the telephone line. In other words, the correction sentence can be provided as voice information to both the originating speaker and the receiving speaker. This allows the erroneous utterance portion to be corrected simultaneously for both the originating speaker and the receiving speaker.

As described above, the voice processing unit 121A connects a telephone line and processes the voices of both speakers who are engaged in a call, and processes the voices so that it is possible to distinguish which speaker is the subject of the processing. Simply put, when a communication line is connected between telephone terminal 3 and outside telephone terminal 7, the voice processing unit 121A processes the voices so that it is possible to distinguish whether they are the voices from telephone terminal 3 or the voices from outside telephone terminal 7. Therefore, it is possible to distinguish whether the ambiguous utterance portion for which the correction sentence was created is the part of the voice from telephone terminal 3 or the part of the voice from outside telephone terminal 7.

The correction providing unit 126 can therefore convert the correction sentence (text data) created by the correction sentence creating unit 125 into audio information and provide it only to the speaker who made the erroneous statement. In this case, the speaker who made the erroneous statement can correct the error in their own statement using their own voice and provide it to the other speaker. In this case, a softer response can be achieved, in that the speaker who made the erroneous statement feels convinced and the other speaker can easily accept the correction.

<Processing in the communication support system according to the first embodiment>
3 is a sequence diagram for explaining the process of the call support system configured using the telephone control device 1A of the first embodiment. As described above, it is assumed that a call line is connected between the telephone terminal 3 and the outside line telephone terminal 7 by making a call from the telephone terminal 3 to the outside line telephone terminal 7, or by making a call from the outside line telephone terminal 7 to the telephone terminal 3, and a call is started (step S1).

As explained using FIG. 1, telephone terminal 3 is a telephone terminal subordinate to telephone control device 1A, so all voice communication between telephone terminal 3 and outside telephone terminal 7 is sent and received via telephone control device 1A. In other words, telephone control device 1A relays all voice communication between telephone terminal 3 and outside telephone terminal 7. For this reason, in telephone control device 1A, transfer of voice communication and voice recognition are started under the control of control unit 102 (step S2).

Specifically, in step S2, the call voice from the outside telephone terminal 7 received through the connection end 101T and the telephone network I/F 101 is transferred (sent) to the telephone terminal 3 through the extension I/F 107 and the connection end 107T. Also, in step S2, the call voice from the telephone terminal 3 received through the connection end 107T and the extension I/F 107 is transferred (sent) to the outside telephone terminal 7 through the telephone network I/F 101 and the connection end 101T. Furthermore, in step S2, under the control of the control unit 102, the voice recognition unit 1211 of the voice processing unit 121A functions to start a process of converting the call voice from the telephone terminal 3 and the call voice from the outside telephone terminal 7 into text data.

At the same time, the control unit 102 controls the ambiguous utterance detection unit 1212 and keyword extraction unit 1213 of the voice processing unit 121A to detect ambiguous utterance parts and extract search keywords (step S3). In step S3, the ambiguous utterance detection unit 1212 receives text data from the voice recognition unit 1211, detects phrases and words, and detects ambiguous utterance parts by referring to the ambiguous utterance dictionary 105A. As described above, ambiguous utterance parts include parts that include ambiguous expressions such as "probably," "I think," "if I remember correctly," and "probably," as well as parts that include numbers such as "xx minutes," "xx hours," and "△△ yen."

In step S3, the keyword extraction unit 1213 assumes that there is text data corresponding to the voice of the call, such as, for example, "I think the next national holiday after Mountain Day is probably Autumnal Equinox Day." In this case, the part "I think it's probably Autumnal Equinox Day" is extracted as an ambiguous utterance part. Therefore, the keyword extraction unit 1213 also takes into account the part immediately before the ambiguous part, and extracts the phrase "The next national holiday after Mountain Day is," which is equivalent to "Autumnal Equinox Day," as a search keyword.

In step S3, the keyword extraction unit 1213 assumes that there is text data corresponding to the voice of the call, such as, for example, "The time required from Machida to Hashimoto is 5 minutes." In this case, the part "It takes 5 minutes" is extracted as an ambiguous utterance part. Therefore, the keyword extraction unit 1213 also takes into consideration the part immediately before the ambiguous part, and extracts the phrase "The time required from Machida to Hashimoto," which is the part equivalent to "5 minutes," as a search keyword.

Next, under the control of the control unit 102, the search destination determination unit 122 functions to refer to the search destination determination dictionary 106 and perform a process of determining the search destination based on the search keywords extracted by the keyword extraction unit 1213 (step S4). Specifically, the search destination determination unit 122 determines whether the search destination should be the internal information server 5(1), ... on the LAN 4, or the information providing server 8(1), ... on the IP network 62.

For example, in the case of the above-mentioned search keyword "What is the next national holiday after Mountain Day?", words such as "Mountain Day" and "national holiday" are general words and are not phrases that are particularly used internally (within a company), so it is determined that the information providing server 8 (1), ... on the IP network 62 is the search destination. In contrast, if the words included in the search keyword are phrases that are particularly used internally (within a company), such as "product number 1234" or "sales representative of XXX Co., Ltd.", it is determined that the internal information server 5 (1), ... on the LAN 4 is the search destination.

In addition, if the search keyword is "travel time from Machida to Hashimoto," it is possible to specify the web page itself to be searched (inquiry destination) based on the search keyword, such as searching a specific railway company's web page on IP network 62. Similarly, if the search keyword is information indicating a customer, such as "product number 1234" or "sales representative of XXX Co., Ltd.", it is also possible to specify the database itself to be searched, such as a product information DB (Data Base) or customer information DB on LAN 4.

Next, under the control of the control unit 102, the search execution unit 123 functions to execute a search using the search keyword extracted by the keyword extraction unit 1213 for the search destination determined by the search destination determination unit 122 (step S5). When the search destination is the information providing server 8 (1), ... on the IP network 62, the search execution unit 123 uses a specified browser (web page viewing software) to input the search keyword, and executes a search using the information providing server 8 (1), ... on the IP network 62 as the search destination. As a result, for example, when the search keyword "What is the next national holiday after Mountain Day?" is used, the search result "Respect for the Aged Day" is obtained. Also, when the search keyword "How long does it take to get from Machida to Hashimoto?" is used, the search result "11 minutes for rapid train, 14 minutes for local train" is obtained.

Furthermore, when the search destination is the internal information server 5(1), ... on the LAN 4, the search execution unit 123 executes a specified search program and inputs a search keyword, and performs a search on the internal information server 5(1), ... on the LAN 4 as the search destination. As a result, to give a specific example, when the search keyword "product number 1234" is used, the product information DB is referenced and detailed information such as the specifications of product number 1234 is obtained. When the search keyword "sales representative of XX Co., Ltd." is used, the customer information DB is referenced and the name of the sales representative of XX Co., Ltd. is obtained, such as "Sales Division 1, Suzuki Taro."

Next, under the control of the control unit 102, the truth/false judgment unit 124 functions to compare the ambiguous statement portion detected by the ambiguous statement detection unit 1212 with the search result acquired by the search execution unit 123, and judges whether the content of the ambiguous statement portion is correct or incorrect (step S6). Specifically, when the ambiguous statement portion is "Probably/Autumnal Equinox/I think," the search result is "Respect for the Aged Day" as described above, so it is judged that the ambiguous statement portion is incorrect. Also, when the ambiguous statement portion is "5 minutes," the search result is "11 minutes for rapid train, 14 minutes for local train" as described above, so it can be judged that the ambiguous statement portion is incorrect.

If it is determined in step S6 that the ambiguous utterance is not an error, then no subsequent correction is necessary, and the process for that ambiguous utterance is terminated, and the process moves on to the next ambiguous utterance. Assume that it is determined to be an error in the step S6 determination process. In this case, under the control of the control unit 102, the correction statement creation unit 125 functions to perform a process for creating a correction statement (step S7). In step S7, the correction statement creation unit 125 creates a correction statement to correct the content of the detected ambiguous utterance, taking into account the search results of the search execution unit 123 and information such as the search keywords extracted by the keyword extraction unit 1213.

For example, in the case of the former example described above, a correction sentence such as "The next national holiday after Mountain Day is Respect for the Aged Day" would be created. In the case of the latter example described above, a correction sentence such as "The travel time from Machida to Hashimoto is 11 minutes by rapid train and 14 minutes by local train" would be created. Thereafter, under the control of the control unit 102, the correction providing unit 126 functions to convert the correction sentence (text data) created by the correction sentence creating unit 125 into voice data and provide it to the speaker connected to the telephone line (step S8).

In this case, the voice data corresponding to the correction sentence (corrected voice data) is provided to both speakers by the correction providing unit 126 when, for example, the control unit 102 detects that the call voices of both speakers have been interrupted (steps S9 and S10). This is to ensure that the voice data corresponding to the correction sentence is provided appropriately without interrupting the call. As a result, if the ambiguous utterance portion detected in step S3 contains an error, a correction can be automatically made to both speakers by the function of the telephone control device 1A.

After this, when the correction providing unit 126 finishes sending the voice data of the correction sentence, the control unit 102 erases the voice data of the correction sentence and returns to the normal call (step S11). This allows the transmission and reception of the voice of the call to continue without interruption (step S12), and the process from step S2 described above is repeated.

In this way, the telephone control device 1A of the first embodiment can analyze the voices of both speakers who are connected over a telephone line and make post-correction (correction of ambiguous statements). This allows mistakes to be automatically corrected to the correct information without being left as errors, preventing inaccurate statements from being made and causing inconvenience to the other party. This prevents disadvantages from occurring due to inaccurate statements.

[Second embodiment]
<Configuration example of telephone control device 1B according to the second embodiment>
The telephone control device 1B of the second embodiment has a function of analyzing the voices of both speakers who are connected to a telephone line and performing advance supplementation (supplementation of unclear utterances). Fig. 4 is a block diagram for explaining a configuration example of the telephone control device 1B to which the second embodiment of the communication support device of the present invention is applied. In the telephone control device 1B shown in Fig. 4, parts configured similarly to the telephone control device 1A of the first embodiment explained using Fig. 2 are given the same reference numerals, and detailed explanations of these parts are omitted to avoid duplication.

As shown in FIG. 4, the telephone control device 1B of the second embodiment includes an unknown situation dictionary 105B. The unknown situation dictionary 105B is created in a recording device such as an HDD or SSD, and holds various dictionary data for detecting unclear utterances in the voices of both speakers who are talking over a connected telephone line. Examples of dictionary data include expressions for when you can't find the words, such as "Um, um, ...", "Um, ...", "What was it again?" and "How many minutes was it again?", as well as expressions for asking the other party questions.

The conversation support processing unit 120B is the unit that realizes the function of analyzing the voices of both speakers who are connected via telephone lines and performing advance supplementation (supplementation of unclear utterances). As shown in FIG. 2, the conversation support processing unit 120B includes a voice processing unit 121B that is made up of a voice recognition unit 1211, an unclear utterance detection unit 1214, and a keyword extraction unit 1215. Furthermore, the conversation support processing unit 120B includes a search destination determination unit 122, a search execution unit 123, a search success/failure determination unit 127, a supplementary sentence creation unit 128, and a supplement provision unit 129.

As described above, the voice recognition unit 1211 connects a telephone line and converts the voices of both speakers who are speaking into text data. In this case, the voice recognition unit 1211 converts the voices of the telephone call from the telephone terminal 3 and the voices of the telephone call from the external telephone terminal 7 into text data so that they can be distinguished from each other. The unclear utterance detection unit 1214 analyzes the text data converted from the voices of the speakers by dividing them into phrases and words, and detects the unclear utterance portion by referring to the unclear situation dictionary 105B. In addition, when the voices of the telephone call (utterance) are interrupted midway, such as "The product number is...", and then silence occurs for a predetermined period of time or more (for example, 2 seconds or more), the unclear utterance detection unit 1214 detects the relevant portion as the unclear utterance portion.

The keyword extraction unit 1215 extracts search keywords for searching the content of the unclear utterance portion detected by the unclear utterance detection unit 1214 from the text data converted by the voice recognition unit 1211. The search destination determination unit 122 refers to the search destination determination dictionary 106 using the search keywords extracted by the keyword extraction unit 1215, and determines whether to search the internal information server 5(1), ... or the information providing server 8(1), ... on the IP network 62.

The search execution unit 123 performs processing to execute a search using the search keywords extracted by the keyword extraction unit 1215, with the information stored in the search destination determined by the search destination determination unit 122 as the search target. In this case, if the search destination is the internal information server 5(1), ..., a specified search program is executed, and information matching the search keywords (search results) is obtained from the information stored in the internal information server 5(1), .... Also, if the search destination is an information providing server 8(1), ... on the IP network 62, the search execution unit 123 executes a specified browser (web page viewing software), executes a search using the search keywords, and obtains information matching the search keywords (search results).

The search success/failure determination unit 127 determines whether or not an answer corresponding to the unclear statement portion has been found based on the search result by the search execution unit 123. The supplementary sentence creation unit 128 functions according to the determination result by the search success/failure determination unit 127. That is, if the determination result by the search success/failure determination unit 127 indicates that an answer corresponding to the unclear statement portion has not been found, the supplementary sentence creation unit 128 creates a supplementary sentence notifying the user that an answer to the unclear portion has not been obtained. Also, if the determination result by the search success/failure determination unit 127 indicates that an answer corresponding to the unclear statement portion has been found, the supplementary sentence creation unit 128 creates a supplementary sentence including the search result by the search execution unit 123.

The supplementary sentence providing unit 129 converts the supplementary sentence (text data) created by the supplementary sentence creating unit 128 into audio information and provides it to the speaker connected to the telephone line via the telephone line. In other words, the supplementary sentence can be provided as audio information to both the calling speaker and the called speaker. This makes it possible to provide supplementary information on unclear utterances to both the calling speaker and the called speaker at the same time.

As described above, the voice processing unit 121B connects a telephone line and processes the voices of both speakers in a call, and processes the voices in a manner that makes it possible to distinguish which speaker's voice is being processed. Simply put, for example, when a communication line is connected between telephone terminal 3 and outside telephone terminal 7, the voice processing unit 121B processes the voices in a manner that makes it possible to distinguish whether they are the voices from telephone terminal 3 or the voices from outside telephone terminal 7. For this reason, it is possible to distinguish whether the unclear utterance portion for which the supplementary sentence was created is the part of the voice from telephone terminal 3 or the part of the voice from outside telephone terminal 7.

The supplementary provision unit 129 can therefore convert the supplementary sentence (text data) created by the supplementary sentence creation unit 128 into audio information and provide it only to the speaker who made the unclear statement. In this case, the speaker who made the unclear statement can supplement the unclear part of their own statement with their own voice and provide it to the other speaker. In this case, a softer response can be achieved, in that the speaker who made the unclear statement feels convinced and the other speaker can easily accept the supplement.

<Processing in the communication support system according to the second embodiment>
Fig. 5 is a sequence diagram for explaining the process of the call support system configured using the telephone control device 1B of the second embodiment. In Fig. 5, it is assumed that a call line is connected between the telephone terminal 3 and the outside telephone terminal 7 by making a call from the telephone terminal 3 to the outside telephone terminal 7, or by making a call from the outside telephone terminal 7 to the telephone terminal 3, and a call is started (step S21).

As explained using FIG. 1, telephone terminal 3 is a telephone terminal under telephone control device 1B, so all call voice between telephone terminal 3 and outside telephone terminal 7 is sent and received via telephone control device 1B. That is, telephone control device 1B relays all call voice between telephone terminal 3 and outside telephone terminal 7. For this reason, in telephone control device 1B, transfer of call voice and voice recognition are started under the control of control unit 102 (step S22). In this way, the process of step S21 in FIG. 5 is the same process as the process of step S1 shown in FIG. 3, and the process of step S22 in FIG. 5 is the same process as the process of step S2 shown in FIG. 3.

At the same time, the control unit 102 controls the unclear utterance detection unit 1214 and the keyword extraction unit 1215 of the voice processing unit 121B to detect unclear utterance parts and extract search keywords (step S23). In step S23, the unclear utterance detection unit 1214 receives text data from the voice recognition unit 1211, detects phrases and words, and detects unclear utterance parts by referring to the unknown situation dictionary 105B. As described above, the unclear utterance parts are parts that include expressions when the words cannot be found, such as "Um, um, ...", "What was it?", and "How many minutes was it?", and parts that include expressions to ask the other party. Furthermore, parts where the voice (utterance) of a call is interrupted midway and then silence occurs for a predetermined time or more (for example, 2 seconds or more) are also detected as unclear utterance parts.

For example, in a case where a voice message includes the following statement: "The model number of the adapter delivered to Mr./Ms. XX last month is, um, um, ...", the portion "um, um, ..." is extracted by the unknown utterance detection unit 1214 as an unclear utterance. In this case, the keyword extraction unit 1215 takes into account the portion immediately preceding the unclear utterance and extracts search keywords such as "Mr./Ms. XX", "Delivered last month", and "Adapter model number". In addition, in a case where a voice message includes the following statement: "How many minutes is the flight time from Haneda to Fukuoka?", the portion "How many minutes is it?" is detected by the unknown utterance detection unit 1214 as an unclear utterance. In this case, the keyword extraction unit 1215 takes into account the portion immediately preceding the unclear utterance and extracts search keywords such as "Flight time from Haneda to Fukuoka".

Next, under the control of the control unit 102, the search destination determination unit 122 functions to refer to the search destination determination dictionary 106 based on the search keywords extracted by the keyword extraction unit 1213 and perform a process to determine the search destination (step S24). In step S24, the search destination determination unit 122 refers to the search destination determination dictionary 106 based on the search keywords extracted by the keyword extraction unit 1213 and determines whether the search destination should be the internal information server 5 (1), ... or the information providing server 8 (1), ... on the IP network 62.

For example, if the words included in the search keyword are phrases that are used especially internally (within a company), such as the above-mentioned "Mr./Ms. XX," "Delivered last month," and "Adapter model number," then it is determined that the internal information server 5 (1), ... on the LAN 4 is the search destination. In contrast, in the case of the above-mentioned search keyword "Flight time from Haneda to Fukuoka," words such as "Haneda," "Fukuoka," and "Flight time" are general words and are not phrases that are used especially internally (within a company). For this reason, in this case, it is determined that the information providing server 8 (1), ... on the IP network 62 is the search destination.

In addition, when the search keyword is information indicating a customer, such as "Mr./Ms. XX" or "adapter model number," it is possible to specify the database itself to be searched, such as a customer information DB or product information DB on LAN 4. Similarly, when the search keyword is "flight time from Haneda to Fukuoka," it is also possible to specify the web page itself to be searched (inquiry destination) based on the search keyword, such as searching the web page of a specific airline on IP network 62.

Next, under the control of the control unit 102, the search execution unit 123 functions to execute a search for the search destination determined by the search destination determination unit 122 using the search keyword extracted by the keyword extraction unit 1215 (step S25). When the search destination is an information providing server 8(1), ... on the IP network 62, the search execution unit 123 uses a specified browser (web page viewing software) to input the search keyword, and executes a search for the information providing server 8(1), ... on the IP network 62 as the search destination. As a result, for example, when the search keyword "flight time from Haneda to Fukuoka" is used, the search result "about 1 hour and 50 minutes" is obtained.

Furthermore, when the search destination is the internal information server 5 (1), ... on the LAN 4, the search execution unit 123 executes a specified search program and inputs search keywords, and performs a search on the internal information server 5 (1), ... on the LAN 4 as the search destination. As a specific example, assume that the search keywords "Mr. XX", "Delivered last month", and "Adapter model number" are used. In this case, the transaction information DB is referenced, and a search result of, for example, "ADP1234", which is the "Adapter model number" "Delivered last month" to the customer "Mr. XX", is obtained.

Next, under the control of the control unit 102, the search success/failure determination unit 127 functions to determine whether the search performed by the search execution unit 123 is successful (step S26). Specifically, the search execution unit 123 determines whether a search result has been obtained. Next, under the control of the control unit 102, the supplementary sentence creation unit 128 functions to perform a process of creating a supplementary sentence (step S27). If the search success/failure determination unit 127 determines in step S26 that no search result has been obtained (search unsuccessful), the supplementary sentence creation unit 128 creates a supplementary sentence such as "Search failed" in step S27. Conversely, if the search success/failure determination unit 127 determines in step S26 that a search result has been obtained (search successful), the supplementary sentence creation unit 128 creates a supplementary sentence based on the result of the search process performed by the search execution unit 123 in step S27.

To give a specific example, if the search keyword "flight time from Haneda to Fukuoka" is used and the search result is "approximately 1 hour 50 minutes," a supplemental sentence such as "Flight time from Haneda to Fukuoka is approximately 1 hour 50 minutes" is created. Similarly, if the search keywords "Mr./Ms. XX," "Delivered last month," and "Adapter model number" are used and the search result is "ADP1234," a supplemental sentence such as "The model number of the adapter delivered to Mr./Ms. XX last month is ADP1234" is created.

After this, under the control of the control unit 102, the supplementary provision unit 129 functions to convert the supplementary sentence (text data) created by the supplementary sentence creation unit 128 into audio data and provide it to the speakers connected on the telephone line (step S28). In this case, for example, if the control unit 102 detects that the voice of both speakers has been interrupted, the supplementary provision unit 129 provides it to both speakers (steps S29, S30). This is to provide appropriate audio data corresponding to the supplementary sentence without disrupting the call. As a result, for the unclear utterance portion detected in step S23, the function of the telephone control device 1B can automatically provide both speakers with information that supplements (complements) that portion of the utterance.

After this, when the transmission of the voice data of the supplementary sentence from the supplementary provision unit 129 is completed, the control unit 102 erases the voice data of the supplementary sentence and returns to the normal call (step S31). After the processing of step S31, the voice processing unit 121B functions, the voice recognition unit 1211 converts the call voice into text data, and the unclear statement detection unit 1214 functions to extract a portion indicating an evaluation of the provided supplementary sentence. For example, if negative words such as "That's not it," "It's not right," or "That doesn't make sense," are extracted immediately after the provision of the supplementary sentence, the control unit 102 returns to the processing from step S25, searches again, and provides additional supplementation using another search result (step S32).

In contrast to this, for example, if positive words such as "Ah, that's right," "I see," or "Finally, I feel relieved" are extracted immediately after the supplementary text is provided, the search for the current unclear portion is terminated (step S33). Also, if words indicating that no supplement is necessary such as "I will look into this matter and get back to you later" or "Maybe another time" are extracted immediately after the supplementary text is provided, the search for the current unclear portion is terminated (step S33). After this, the transmission and reception of voice communication continues (step S34), and the process from step S23 described above is repeated.

Note that in steps S32 and S33, although not shown, for example, a "negative phrase dictionary" stored in the storage device 103 is used, and in step S33, although not shown, for example, a "positive phrase dictionary" stored in the storage device 103 is used.

In this way, the telephone control device 1B of the second embodiment can analyze the voices of both speakers who are connected to the telephone line and can perform advance supplementation (supplementation of unclear utterances). As a result, even if something unclear is said during a call and the conversation does not progress, an answer is automatically provided for the unclear utterance, preventing delays in the conversation and allowing the call (conversation) to proceed smoothly.

[Third embodiment]
<Configuration Example of Telephone Control Device 1C of Third Embodiment>
The telephone control device 1C of the third embodiment has a function of analyzing the voices of both speakers who are connected to a telephone line and performing preventive measures (overwriting inappropriate remarks). Fig. 6 is a block diagram for explaining a configuration example of the telephone control device 1C to which the third embodiment of the communication support device of the present invention is applied. In the telephone control device 1C shown in Fig. 6, parts configured similarly to the telephone control device 1A of the first embodiment described using Fig. 2 are given the same reference numerals, and detailed explanations of those parts are omitted.

As shown in FIG. 6, the telephone control device 1C of the third embodiment includes an inappropriate word dictionary 105C. The inappropriate word dictionary 105C is created in a recording device such as an HDD or SSD, and holds various dictionary data for detecting inappropriate utterances in the voices of both speakers who are connected to a telephone line and making a call. The inappropriate word dictionary 105C stores various words that may make the other party feel uncomfortable, such as words that make the other party feel insulted, discriminatory words, inappropriate words, and words that go against public order and morals.

The conversation support processing unit 120C is the part that realizes the function of analyzing the voices of both speakers who are connected to the telephone line and preventing inappropriate remarks in advance (overwriting inappropriate remarks). As shown in FIG. 2, the conversation support processing unit 120C includes a voice processing unit 121C that is made up of a voice recognition unit 1211, an inappropriate remark detection unit 1216, and an inappropriate remark overwriting unit 1217. Furthermore, the conversation support processing unit 120C includes a guidance creation unit 131 and a guidance provision unit 132.

In the third embodiment, when an inappropriate remark is included in a call voice from a subordinate telephone terminal 3, the telephone control device 1C does not provide that portion to the outside telephone terminal 7. Furthermore, the telephone control device 1C provides a guidance message calling attention to the telephone terminal 3 that is the sender of the call voice including the inappropriate remark. Therefore, even if an inappropriate remark is included in a call voice from the outside telephone terminal 7, this is provided to the telephone terminal 3 as is.

That is, the voice recognition unit 1211 can distinguish between the call voice from the telephone terminal 3 and the call voice from the external telephone terminal 7. Therefore, in the telephone control device of this third embodiment, a call line is connected and a process is performed to convert the call voice from the telephone terminal 3 that makes the call into text data. The inappropriate remark detection unit 1216 analyzes the text data converted from the call voice from the speaker of the telephone terminal 3 by dividing it into phrases and words, and detects inappropriate words by referring to the inappropriate word dictionary 105C.

The inappropriate remarks overwriting unit 1217 replaces the portion of the telephone call audio corresponding to the inappropriate remarks detected by the inappropriate remarks detection unit 1216 with silence in the telephone control device 1C of this embodiment, and transmits this to the outside telephone terminal 7, which is the other party. In this way, the transmitted and received telephone call audio is always transmitted to the other party via the audio processing unit 121C. The guidance creation unit 131 creates a guidance message that calls attention to the speaker of the telephone terminal 3, which is the source of the telephone call audio in which the inappropriate remarks were detected. The guidance message is, for example, "An inappropriate remark was made. The audio information for that portion has not been transmitted to the other party."

The guidance providing unit 132 converts the guidance message (text data) created by the guidance creating unit 131 into voice information and provides it only to the speaker of the telephone terminal 3 who made the inappropriate remark. In other words, the guidance providing unit 132 provides the guidance message (text data) created by the guidance creating unit 131 to the subordinate telephone terminal 3 connected to the telephone line via the extension I/F 107 and the connection end 107T.

In the telephone control device 1C of the third embodiment, inappropriate remarks are detected from the call voice from the subordinate telephone terminal 3. However, the voice processing unit 121C of the telephone control device 1C of the third embodiment can also connect a call line, process the call voices from both speakers of the call, and process the call voice of each speaker in a manner that makes it possible to distinguish which speaker is the target of processing. Therefore, inappropriate remarks can be detected from the call voice from the outside telephone terminal 7 in the same way as when the call voice from the telephone terminal 3 is processed, and the call voice in question is not provided to the telephone terminal 3. In this case, a guidance message notifying that an inappropriate remark has been made can also be formed and provided to the outside telephone terminal 7.

<Processing in the communication support system according to the third embodiment>
Fig. 7 is a sequence diagram for explaining the process of the call support system configured using the telephone control device 1C of the third embodiment. In Fig. 7, it is assumed that a call line is connected between the telephone terminal 3 and the outside telephone terminal 7 by making a call from the telephone terminal 3 to the outside telephone terminal 7, or by making a call from the outside telephone terminal 7 to the telephone terminal 3, and a call is started (step S41).

As explained using FIG. 1, telephone terminal 3 is a telephone terminal subordinate to telephone control device 1C, and therefore all call voice between telephone terminal 3 and outside telephone terminal 7 is sent and received via telephone control device 1C. In other words, telephone control device 1C relays all call voice between telephone terminal 3 and outside telephone terminal 7. For this reason, in telephone control device 1C, transfer of call voice and voice recognition are started under the control of control unit 102 (step S42).

In the telephone control device 1C of the third embodiment, for example, the control unit 102 functions to extract information about the speaker of the outside telephone terminal 7, which is the other end of the call, from the content of the greeting, etc., based on the text data converted from the call voice by the voice recognition unit 1211 (step S43). In addition, in step S43, the control unit 102 also uses the telephone number assigned to the outside telephone terminal 7, which is the other end of the call, to extract information about the speaker of the outside telephone terminal 7. When the outside telephone terminal 7 is the caller, the caller telephone number provided by the outside telephone terminal 7 can be used as the telephone number of the outside telephone terminal 7, and when the outside telephone terminal 7 is the callee, the callee telephone number provided by the caller telephone terminal 3 can be used.

The information about the speaker of the external telephone terminal 7 extracted in step S43 is used in the process of extracting inappropriate remarks. For example, if the speaker of the external telephone terminal 7 is a customer, inappropriate remarks must be detected at a high level so as not to be rude. Also, if the speaker of the external telephone terminal 7 is a supplier, particular attention must be paid to harassing remarks. In this way, the information about the speaker of the external telephone terminal 7 is also taken into consideration, since the category of remarks that require attention may differ depending on who the speaker of the external telephone terminal 7 is. After the process of step S43, the voice recognition performed by the voice recognition unit 1211 of the telephone control device 1C may be limited to only the call voice from the telephone terminal 3 subordinate to the telephone control device 1C.

Then, the voice processing unit 121C of the telephone control device 1C processes the call voice from the telephone terminal 3 (step S44). Therefore, for the call voice from the telephone terminal 3, the voice recognition unit 1211 functions to convert the call voice into text data, and for this text data, the inappropriate remark detection unit 1216 refers to the inappropriate phrase dictionary 105C to detect inappropriate remarks (step S45). Note that in step S45, the inappropriate remark detection unit 1216 can adjust the level of what is considered inappropriate depending on who is speaking at the outside telephone terminal 7, which is the other party.

For this reason, the dictionary data registered in the inappropriate phrases dictionary 105C is provided with information indicating the level, for example, so that inappropriate remarks can be detected according to the level. For example, if the other party is a customer, inappropriate remarks can be detected at a high level of strictness across the entire range, and if the other party is a supplier, inappropriate remarks can be detected at a strict level only with respect to the scope of harassment.

In step S45, it is assumed that an inappropriate remark portion is detected in the call audio from telephone terminal 3. In this case, the inappropriate remark detection unit 1216 provides information indicating the inappropriate remark portion of the call audio from the telephone terminal to the inappropriate remark overwriting unit 1217. Here, the information indicating the inappropriate remark portion of the call audio is, for example, time information starting from the start of the call, or address information (pointer information) assigned from the beginning of the call audio.

Based on the information from the inappropriate remarks detection unit 1216, the inappropriate remarks overwriting unit 1217 overwrites the inappropriate remarks in the voice of the call from the telephone terminal 3 with silence, and transmits this overwritten voice of the call to the outside telephone terminal 7, which is the other end of the call (step S46). As a result, the voice of the call from the telephone terminal 3 in which the inappropriate remarks have been overwritten with silence is provided to the outside telephone terminal 7, which is the other end of the call (step S47).

Furthermore, in the telephone control device 1C, the guidance creation unit 131 functions to generate a guidance message (text data) notifying the user that an inappropriate comment has been made (step S48).

The guidance message created by the guidance creation unit 131 is converted into audio information by the guidance provision unit 132 and transmitted to the telephone terminal 3 that is the sender of the inappropriate remarks and is currently speaking over a connected telephone line (step S49). As a result, the guidance message is provided to the telephone terminal 3 (step S50). This allows the speaker of the telephone terminal 3 to recognize that they have made an inappropriate remark and to be careful about future remarks.

The control unit 102 then controls each unit to return to a normal call (step S51). This allows the transmission and reception of voice communication to continue uninterrupted (step S52), and the process from step S44 described above is repeated. In this way, a speaker using a telephone terminal 3 under the telephone control device 1C can recognize from the guidance message that he or she has made an inappropriate remark, without notifying the other party of the inappropriate remark, and can be careful about future remarks. This prevents the other party from being unintentionally made to feel uncomfortable during a call.

In the third embodiment, inappropriate remarks are detected and replaced with silence only for the call voice from telephone terminal 3 under the telephone control device 1C, but this is not limited to the above. As described above, inappropriate remarks can also be detected for call voice from external telephone terminal 7 in the same way as when the call voice from telephone terminal 3 is the processing target, and the call voice in question can be prevented from being provided to telephone terminal 3. In this case, a guidance message can also be formed to notify the external telephone terminal 7 that an inappropriate remark has been made.

Therefore, when making an internal call, for example, when a call is made between telephone terminal 3(1) and telephone terminal 3(3), it is possible to detect inappropriate remarks in the call voice from both parties, replace those parts of the call voice with silence, and provide the silenced parts to the other party. Even in this case, it is possible to determine which telephone terminal made the inappropriate remark. The sender can be identified based not only on differences in the call paths, but also on information indicating the sender that is added to the call voice (voice data). This makes it possible to provide a guidance message to the telephone terminal of the speaker who made the inappropriate remark.

In addition, the guidance message is not limited to the above, and it is also possible to use a guidance message to inform the speaker which words uttered by the speaker are inappropriate, for example, "XXX... is an inappropriate statement." This allows a speaker who has made an inappropriate statement without realizing it to clearly understand what was inappropriate and to be conscious of not making that statement in the future.

[Application to telephone conferences and online meetings]
As described above, the present invention is not only a telephone control device that relays a call between two people, but also a conference in which two or more speakers participate, such as a telephone conference or an online conference. A telephone conference can be realized if the telephone control device or cloud PBX is provided with a telephone conference function. In the case of an online conference, it is realized by a conference server on the IP network 62.

For this reason, the conversation support processing units 120A, 120B, and 120C of the telephone control devices 1A, 1B, and 1C described above can be provided in a telephone control device or cloud PBX equipped with a telephone conference function, or in a conference server on the IP network 62, and the call voices from each speaker can be processed. This makes it possible to use the following functions even in a telephone conference or online conference in which two or more speakers participate: (1) post-correction (correction of ambiguous remarks), (2) advance supplementation (supplementation of unclear remarks), and (3) advance prevention (overwriting of inappropriate remarks).

[Effects of the embodiment]
According to the present invention, when a two-party call, a telephone conference, or an online conference is held, each speaker (participant) can be appropriately supported. This improves the convenience of all speakers and improves the quality of the two-party call, the telephone conference, or the online conference.

[Modification]
The telephone control devices of the first, second, and third embodiments described above have been described as having different functions, but are not limited to this. By providing the conversation support processing units 120A, 120B, and 120C, a telephone control device having two functions, namely, (1) post-correction (correction of ambiguous remarks) function, (2) advance supplement (supplement of unclear remarks) function, and (3) advance prevention (overwriting of inappropriate remarks) function, can be realized. Of course, it is also possible to provide two of the functions of (1) post-correction function, (2) advance supplement function, and (3) advance prevention function. The same applies to telephone control devices and cloud PBXs with telephone conference functions, and conference servers for realizing online conferences.

In addition, in the above-mentioned embodiment, it has been explained that the present invention can be applied to telephone control devices, cloud PBXs, conference servers, etc., but the present invention is not limited to these. For example, the present invention can also be applied to a gateway device that connects a company to an IP network. In short, when a call is made between multiple speakers in which voice information is sent and received through various networks, the present invention can be applied to various devices that relay voice information from each speaker.

In the third embodiment described above, inappropriate remarks are described as various words that make the other party feel uncomfortable, such as words that make the other party feel insulted, discriminatory words, inappropriate words, and words that go against public order and morals. However, this is not limited to these. Even if there is a word that does not fall under these words but that you do not want to convey to the other party, you can register it in the inappropriate word dictionary 105C so that it will not be conveyed to the other party. For example, the identity of the other party can be determined from identification information unique to each speaker, such as a telephone number or a connection ID. Therefore, by registering words that the other party may be concerned about or dislike in the inappropriate word dictionary 105C depending on the other party, you can prevent inappropriate words from being conveyed to the other party.

Also, in the first and second embodiments, the identity of the caller can be determined by analyzing identification information unique to each speaker, such as a telephone number or a connection ID, or by analyzing text data of the sounded voice, such as the greeting at the beginning of the call. This analysis can be performed by the ambiguous utterance detection unit 1212 or the unclear utterance detection unit 1214. Therefore, in the telephone control device 1A of the first embodiment, ambiguous utterance parts can be detected, keywords can be extracted, and searches can be performed, taking into account who the caller is. Also, in the telephone control device 1B of the second embodiment, unclear utterance parts can be detected, keywords can be extracted, and searches can be performed, taking into account who the caller is.

In the first embodiment, the correction text (text data) is converted into audio information and provided. In the second embodiment, the supplementary text (text data) is converted into audio information and provided. In the third embodiment, the guidance message is converted into audio information and provided. However, this is not limited to this. If the recipient of the correction text, supplementary text, or guidance message has a function that can receive text data and display it or convert it into audio information and output it, the correction text, supplementary text, or guidance message may be provided to the recipient as text data.

In particular, when an online conference is held using a PC (Personal Computer), tablet PC, or smartphone through a conference server, corrections, supplementary text, and guidance messages are provided to the intended recipient as text data. As a result, the recipient can display the corrections, supplementary text, and guidance messages on the display of the PC, tablet PC, or smartphone being used and provide them to the user. This allows corrections, supplementary text, and guidance messages to be provided to the intended recipient user as text data at the appropriate time without disrupting the audio conference.

[others]
As can be seen from the above description of the embodiment, the function of the voice recognition means in the claims is realized by the voice recognition unit 1211 of the telephone control devices 1A, 1B, and 1C in the embodiment. Also, the function of the detection means in the claims is realized by the ambiguous remark detection unit 1212 of the telephone control device 1A, the unclear remark detection unit 1214 of the telephone control device 1B, and the inappropriate remark detection unit 1216 of the telephone control device 1C. Also, the function of the creation means in the claims is realized by the keyword extraction unit 1213, the search execution unit 123, and the correction sentence creation unit 125 in the telephone control device 1A. Similarly, the function of the creation means in the claims is realized by the keyword extraction unit 1213, the search execution unit 123, and the supplementary sentence creation unit 128 in the telephone control device 1B, and the inappropriate remark overwriting unit 1217 in the telephone control device 1C. The function of the providing means in the claims is realized by the correction providing unit 126 of the telephone control device 1A, the supplement providing unit 129 of the telephone control device 1B, and the inappropriate comment overwriting unit 1217 of the telephone control device 1C, respectively.

The processes performed by the telephone control devices 1A, 1B, and 1C described using the sequence diagrams in Figures 3, 5, and 7 are an embodiment of the call support method of the present invention.

1A, 1B, 1C...telephone control device, 101T...connection end, 101...telephone network I/F, 102...control device, 103...storage device, 104...terminal management file, 105A...ambiguous speech dictionary, 105B...unknown situation dictionary, 105C...inappropriate word dictionary, 106...search destination determination dictionary, 107T...connection end, 107...extension I/F, 108T...connection end, 108...LAN I/F, 109...call control unit, 109S...outgoing call control unit, 109R...incoming call control unit, 110T...connection end, 110...communication I/F, 120A...conversation support processing unit, 121A...voice processing unit, 1211...voice recognition unit, 1212...ambiguous speech detection unit, 1213...keyword extraction unit, 122...search destination determination unit, 1 23...search execution unit, 124...correction judgment unit, 125...correction sentence creation unit, 126...correction control unit, 120B...conversation support processing unit, 1214...unclear remark detection unit, 1215...keyword extraction unit, 127...search success/failure judgment unit, 128...supplement sentence creation unit, 129...supplement provision unit, 120C...conversation support processing unit, 1216...inappropriate remark detection unit, 1217...inappropriate remark overwriting unit, 131...guidance creation unit, 132...guidance provision unit, 2...extension telephone network, 3, 3(1), 3(2), 3(3)...telephone terminal, 4...LAN, 5(1)...internal information server, 6...wide area network, 61...external telephone network, 62...IP network, 7, 7(1)...external telephone terminal, 8(1)...information provision server

Claims (5)

A communication support device that relays voice information from each speaker when a call is made between a plurality of speakers by transmitting and receiving voice information over a network, comprising:
a speech recognition means for converting speech information from each speaker into text data;
a detection means for analyzing the text data from the speech recognition means and detecting a predetermined portion requiring assistance;
a creating means for creating a message corresponding to the predetermined portion when the detecting means detects the predetermined portion, or for creating processed voice information by processing a portion of the voice information corresponding to the predetermined portion;
and a providing means for converting the message into text information or into audio information and providing the converted audio information to a speaker who provided at least the audio information from which the specified portion was detected, and for providing the processed audio information to a speaker other than the speaker who provided the audio information from which the specified portion was detected. The communication support device according to claim 1,
The detection means detects an ambiguous utterance portion,
The creating means includes:
a first extraction means for extracting a search keyword for searching for an accurate content of the ambiguous utterance portion from the text data from the voice recognition means;
a first search means for searching information disclosed on a predetermined network by using the search keyword extracted by the first extraction means;
and a correction message creation means for creating a correction message as the message based on a search result from the first search means,
The communication support device according to claim 1, wherein the providing means provides the correction sentence as text information or by converting the correction sentence into voice information. The communication support device according to claim 1,
The detection means detects an unclear utterance portion,
The creating means includes:
a second extraction means for extracting search keywords for searching for appropriate content for the unclear utterance portion from the text data from the speech recognition means;
a second search means for performing a search for information disclosed on a predetermined network by using the search keyword extracted by the second extraction means;
and a supplemental text creating means for creating a supplemental text as the message based on a search result from the second search means,
The communication support device according to claim 1, wherein the providing means provides the supplementary sentence as text information or by converting the supplementary sentence into audio information. The communication support device according to claim 1,
The detection means detects an inappropriate comment portion,
The creating means includes:
a communication support device which functions as an overwriting means for creating the processed voice information by overwriting a portion of the voice information from the one speaker corresponding to the inappropriate portion with silence or other information. A method for supporting a call used in a call support device that relays voice information from each speaker when a call is made between a plurality of speakers by transmitting and receiving voice information over a network, comprising:
a speech recognition step in which speech recognition means converts speech information from each speaker into text data;
a detection step in which a detection means analyzes the text data converted in the speech recognition step and detects a predetermined portion requiring assistance;
a creating means for creating a message corresponding to the predetermined portion when the predetermined portion is detected in the detecting step, or for creating processed voice information by processing a portion of the voice information corresponding to the predetermined portion;
a providing step in which a providing means converts the message into text information or into voice information and provides the message to a speaker who provided at least the voice information from which the specified portion was detected, or provides the processed voice information to a speaker other than the speaker who provided the voice information from which the specified portion was detected.

Images