KR102147619B1 - Method for managing phone calls and artificial intelligence assistant system for performing the same - Google Patents

Abstract

The present disclosure relates to a method for managing call contents. The method for managing call contents comprises the steps of: receiving voice data generated during a call between a plurality of user terminals; inputting the received voice data into a speech-to-text transcription model of an artificial neural network to generate output text corresponding to the voice data in real time; and storing the output text generated in real time.

Description

How to manage phone calls, and an artificial intelligence assistant system that implements these methods {METHOD FOR MANAGING PHONE CALLS AND ARTIFICIAL INTELLIGENCE ASSISTANT SYSTEM FOR PERFORMING THE SAME}

The present disclosure relates to a method for managing a phone call and an artificial intelligence assistant system for executing the method, and specifically, an output text corresponding to voice data generated during a call between a plurality of user terminals can be generated in real time. It relates to a method and system.

Recently, not only a conventional public switched telephone network (PSTN), but also a telephone service using a voice over Internet protocol (VoIP), which is an Internet-based telephone network, has been widely used. In addition, due to the development of speech recognition technology, a speech recognition service using speech-to-text (STT) transcription technology is being developed.

Meanwhile, a public switched telephone network (PSTN) and a voice Internet protocol (VoIP) have a problem in that the types of transmission and reception data are different. Due to this problem, it is difficult to provide a voice-to-text (STT) transcription service for a telephone connected to a public switched telephone network (PSTN). In addition, in the conventional speech recognition technology, it is difficult to distinguish users, and the speech recognition rate is low.

The present disclosure provides a method for managing a phone call to solve the above problems and an artificial intelligence assistant system for executing the method.

When users send and receive calls using the public switched telephone network (PSTN), they switch to Voice Internet Protocol (VoIP) to connect the phone, and use voice-to-text (STT) transcription technology to store and transfer all phone calls. A method for managing phone calls made and such an artificial intelligence assistant system are provided.

A method of managing a phone call outputting keywords by inputting text information output through a voice-to-text (STT) transcription technology into an artificial neural network keyword extraction model, and such an artificial intelligence assistant system are provided.

The present disclosure may be implemented in a variety of ways, including a method, system, apparatus, or computer program stored on a computer-readable storage medium.

A method for managing a phone call according to an embodiment of the present disclosure includes the steps of receiving voice data generated during a call between a plurality of user terminals, inputting the received voice data into an artificial neural network voice-text transcription model, and And generating the output text corresponding to the data in real time and storing the output text generated in real time.

According to an embodiment, the step of generating an output keyword by inputting at least some text of the stored output text into the artificial neural network keyword extraction model.

According to an embodiment, the method further includes applying an emphasis effect to the generated output keyword and displaying the output keyword to which the emphasis effect is applied.

According to an embodiment, the method further comprises displaying at least one output keyword among the generated output keywords together with information related to a currency.

According to an embodiment, the method further includes generating a summary text using at least one output keyword among the generated output keywords, and displaying the generated summary text together with information related to a currency.

According to an embodiment, the storing of voice data generated during a call between a plurality of user terminals includes receiving first voice data corresponding to a voice from a first user terminal among a plurality of user terminals, and a plurality of users. Including the step of receiving second voice data corresponding to the voice from the second user terminal of the terminal, and generating the output text in real time, the first voice data and the second voice data artificial neural network voice-text transcription And generating a first output text corresponding to the first voice data and a second output text corresponding to the second voice data by inputting into the model.

According to another embodiment of the present disclosure, a computer program stored in a computer-readable recording medium is provided for executing the above-described method of managing call contents on a computer.

According to another embodiment of the present disclosure, the artificial intelligence assistant system relays a call between a plurality of user terminals, and a relay unit configured to receive voice data generated during the relayed call, and the received voice data are transmitted through an artificial neural network voice- It may include a speech recognition unit configured to input into a speech-to-text transcription model, generate output text corresponding to speech data in real time, and store the output text generated in real time.

According to another embodiment of the present disclosure, the speech recognition unit may be further configured to input at least some text of the stored output text into the artificial neural network keyword extraction model to generate the output keyword.

According to another embodiment of the present disclosure, when a call between a plurality of user terminals corresponds to a telephone call through a public switched telephone network, the relay unit transmits a call through the public switched telephone network via Voice over Internet Protocol (Voice over Internet Protocol; VoIP) relays a call between a plurality of user terminals to switch to a phone call via VoIP), receives first voice data corresponding to a voice from a first user terminal among the plurality of user terminals, and receives a second voice data among the plurality of user terminals. It may be further configured to receive second voice data corresponding to voice from the user terminal.

According to some embodiments of the present disclosure, a public switched telephone network (PSTN) telephone between a user terminal and an external user terminal may be converted to voice Internet protocol (VoIP) to connect the telephone. By receiving voice data generated during a call between a user terminal and an external user terminal, an output text corresponding to the received voice data may be generated in real time. In addition, the output text can be used to generate a key keyword or summary text corresponding to the output text.

According to some embodiments of the present disclosure, the user terminal may easily manage a plurality of phone records and contents by displaying the received keyword or summary text together with information related to a phone call.

According to some embodiments of the present disclosure, a phone list related to a specific project, phone contents, and key keywords may be shared with a plurality of user terminals.

According to some embodiments of the present disclosure, it is possible to learn an artificial neural network voice-text transcription model using voices for each of a plurality of users of a plurality of user terminals, and accordingly, a voice recognition technology optimized for each user Can be provided.

According to some embodiments of the present disclosure, when a user of a user terminal cannot answer a phone call, an artificial intelligence interactive automatic answering service may be provided.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned are those who have general knowledge in the technical field belonging to the present disclosure from the description of the claims (hereinafter, referred to as'normal technician') Will be clearly understood.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, where like reference numerals denote like elements, but are not limited thereto.
1 is an exemplary diagram for explaining speech-text transcription and keyword extraction according to an embodiment of the present disclosure.
2 is a block diagram illustrating an internal configuration of an artificial intelligence assistant system according to an embodiment of the present disclosure.
3 is a block diagram illustrating an artificial intelligence assistant system that performs automatic voice-to-text (STT) transcription by connecting a public switched telephone network (PSTN) according to an embodiment of the present disclosure to a voice Internet protocol (VoIP).
4 is a block diagram illustrating an artificial intelligence assistant system for performing automatic voice-to-text (STT) transcription by connecting a public switched telephone network (PSTN) according to another embodiment of the present disclosure to a voice Internet protocol (VoIP).
5 is a block diagram illustrating an artificial intelligence assistant system that performs automatic voice-to-text (STT) transcription in the Internet Protocol (VoIP) according to an embodiment of the present disclosure.
6 is a block diagram illustrating an artificial intelligence assistant system that performs voice-to-text (STT) transcription in a user terminal according to another embodiment of the present disclosure.
7 is a flowchart illustrating a method of performing speech-to-text (STT) transcription according to an embodiment of the present disclosure.
8 is a flow chart illustrating a method of connecting a public switched telephone network (PSTN) to a voice Internet protocol (VoIP) according to an embodiment of the present disclosure.
9 is a flowchart illustrating a method of displaying, tagging, and generating a summary text for a keyword output based on an output text generated from a speech-to-text (STT) transcription according to an embodiment of the present disclosure to be.
10 is a diagram illustrating an example of keyword tagging based on output text corresponding to voice data according to an embodiment of the present disclosure.
11 is an exemplary view illustrating a method of sharing voice and text data with another user terminal according to an embodiment of the present disclosure.
12 is an exemplary diagram for explaining an automatic response and missed message management by an artificial intelligence assistant system when a missed call is received according to an embodiment of the present disclosure.
13 is an exemplary diagram illustrating an artificial neural network model according to an embodiment of the present disclosure.

Hereinafter, with reference to the accompanying drawings, specific details for the implementation of the present disclosure will be described in detail. However, in the following description, when there is a possibility that the subject matter of the present disclosure may be unnecessarily obscure, detailed descriptions of widely known functions or configurations will be omitted.

In the accompanying drawings, the same or corresponding elements are assigned the same reference numerals. In addition, in the description of the following embodiments, overlapping descriptions of the same or corresponding components may be omitted. However, even if description of a component is omitted, it is not intended that such component is not included in any embodiment.

Advantages and features of the disclosed embodiments, and a method of achieving them will become apparent with reference to the embodiments described below together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments make the present disclosure complete, and the present disclosure completely covers the scope of the invention to those skilled in the art. It is only provided to inform you.

The terms used in the present specification will be briefly described, and the disclosed embodiments will be described in detail. The terms used in the present specification have selected general terms that are currently widely used as possible while considering functions in the present disclosure, but this may vary according to the intention or precedent of a technician engaged in a related field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, not the name of a simple term.

In this specification, expressions in the singular include plural expressions, unless the context clearly specifies that they are singular. In addition, plural expressions include expressions in the singular unless clearly specified as plural in context. When it is said that a part in the specification'includes' a certain element, this means that other elements may be further included instead of excluding other elements unless specifically stated to the contrary.

In addition, the terms "module" or "unit" used in the specification mean software or hardware components, and "module" or "unit" performs certain roles. However,'module' or'unit' is not meant to be limited to software or hardware. The'module' or'unit' may be configured to be in an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, the'module' or'unit' refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, It may include at least one of procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. Components and the functions provided within the'module' or'unit' may be combined into a smaller number of components and'module' or'unit', or additional components and'module' or'unit'. Can be further separated.

According to an embodiment of the present disclosure, a'module' or'unit' may be implemented with a processor and a memory. 'Processor' should be interpreted broadly to include general purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some circumstances, a'processor' may refer to an application specific application (ASIC), programmable logic device (PLD), field programmable gate array (FPGA), and the like. 'Processor' refers to a combination of processing devices such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in combination with a DSP core, or any other such combination of configurations. You may. In addition, the'memory' should be broadly interpreted to include any electronic component capable of storing electronic information.

In the present disclosure, the'system' may include at least one of a server device and a cloud device, but is not limited thereto. For example, the system may consist of one or more server devices. As another example, the system may consist of one or more cloud devices. As another example, in the system, a server device and a cloud device may be configured and operated together.

In the present disclosure,'generated in real time' may refer to generating data at the same time as or almost simultaneously with reception. According to an embodiment, generating the output text corresponding to the voice data in real time may refer to generating the output text as soon as the voice data is received by the artificial neural network voice-text transcription model. Here, after receiving the voice data, the input time to the artificial neural network voice-text transcription model, the processing time by the artificial neural network voice-text transcription model, and/or the output time by the artificial neural network voice-text transcription model may be generated. However, the output text generated after these times can be viewed as being generated in real time.

In the present disclosure, the'call content' may include information related to a call performed by the user terminal. According to an embodiment, the'call content' may include arbitrary data transmitted/received during a call, a call record, whether a call is transmitted/received, a call partner name, a call start time and a call time, etc., but is not limited thereto. For example, arbitrary data transmitted and received during a call may include voice data generated during a call between a plurality of terminals, text data corresponding to the voice data, and the like.

In the present disclosure, a'keyword' may be referred to as a combination of words and/or sentences that can represent this within a document set received based on an automatic summarization technique using artificial intelligence. According to an embodiment, the keyword may be output by an artificial neural network keyword extraction model.

In the present disclosure,'tagging' or'tag' may mean that a keyword is extracted in the form of a subject or category related to an image or text and associated with an image or text. More specifically, natural language can contain inherently many ambiguities at the level of vocabulary, syntax and semantics. Therefore, when a keyword is extracted using the raw raw corpus as it is, accurate language information may not be extracted due to ambiguity. Accordingly, this problem can be overcome by removing the ambiguity in the primitive corpus and attaching an additional language, such an additional language may be referred to as a tag. In addition, attaching such a tag to an image or text may be referred to as tagging. According to an embodiment, the tag may be generated by including or processing at least some of the output keywords output by the artificial neural network keyword extraction model.

In the present disclosure,'voice data' may include arbitrary data that can be generated through a call between a plurality of user terminals. For example,'voice data' includes analog/digital signals transmitted and received in a public switched telephone network (PSTN), digital packets transmitted and received in a voice over Internet Protocol (VoIP) network, etc. Can be, but is not limited thereto. In addition,'voice data' may refer to data that can be transmitted/received and/or reproduced in the form of voice by applying a kind of pre-processing from data generated through a call between a plurality of user terminals. In one embodiment, digital packets may be converted into analog/digital signals by a gateway included in the relay communication service provider server. Similarly, analog/digital signals may be converted into digital packets by a gateway included in the relay communication service provider server. In another embodiment of the present disclosure, the gateway may be included in the artificial intelligence assistant system and perform the same function as the gateway included in the relay communication company server.

1 is an exemplary diagram for explaining speech-text transcription and keyword extraction according to an embodiment of the present disclosure. The artificial intelligence assistant system receives the voice data 110 generated during a call between a plurality of user terminals, inputs the received voice data 110 into the artificial neural network voice-text transcription model, and the voice data 110 The corresponding output text 120 can be generated in real time. The output text 120 generated in real time may be stored in the artificial assistant system. Alternatively, the output text 120 may be stored on any storage device that can be accessed by the artificial assistant system. According to an embodiment, the artificial intelligence assistant system may be stored and operated in the user terminal. According to another embodiment, the artificial intelligence assistant system may be stored and operated in one or more server devices capable of communicating with a user terminal. In addition, the artificial neural network voice-text transcription model will be described in detail with reference to FIG. 13 below.

According to an embodiment, the received voice data 110 may be stored in an artificial intelligence assistant system or stored in an arbitrary storage device accessible to the artificial intelligence assistant system, and the stored voice data 110 may be stored in the user terminal. Can be reproduced.

According to an embodiment, as shown, the output text 120 corresponding to the voice data 110 may include information corresponding to the current playback time of the voice data 110, and the voice data 110 The corresponding output text 120 may be stored in chronological order. When the voice data 110 generated during a call in the user terminal is played back, the output text 120 corresponding to the voice data 110 may also be simultaneously output (eg, displayed on the display) to the user terminal, and thus Not limited.

According to an embodiment, when the output text 120 corresponding to the voice data 110 is generated in real time during a phone call, the output text 120 being generated in real time may be output to the user terminal. Here, the user can check whether an error exists in the output text 120, which is a result of the speech-text transcription model, through the user terminal. Furthermore, the user may modify or supplement the output text 120 through the artificial intelligence assistant system, and the modified or supplemented output text 120 may be stored.

According to an embodiment, as shown, a specific time point of the voice data 110 may be played back through the voice data playback window in the user terminal. According to another embodiment, although not shown, the output text 120 corresponding to the time point at which the voice data 110 is currently being reproduced may be highlighted in the output text window of the user terminal, but is not limited thereto. In another embodiment, a user's selection for a specific output text window is received from the user terminal, and a specific time point of the voice data 110 corresponding to the time point of the specific output text window may be reproduced.

The artificial intelligence assistant system may generate output keywords 130 and 140 by inputting at least some text of the stored output text 120 into the artificial neural network keyword extraction model. Here, the artificial neural network keyword extraction model will be described in detail with reference to FIG. 13. According to an embodiment, when the output text 120 is input to the artificial neural network keyword extraction model, a part of the output text 120 that can be inferred as an important keyword by the artificial neural network keyword extraction model may be generated as an output keyword. have. For example, as shown in FIG. 1, a date and time of'Next Tuesday 3 pm' and a place of'Gangnam Station' among the output text 120 may be generated as output keywords 130 and 140.

According to an embodiment, the artificial intelligence assistant system may apply an emphasis effect to an output keyword generated by the artificial neural network keyword extraction model and display the output keyword to which the emphasis effect is applied. For example, the output keywords 130 and 140 may be highlighted and displayed as shown.

According to another embodiment, the artificial intelligence assistant system may display at least one output keyword among output keywords generated by the artificial neural network keyword extraction model together with information related to a call. For example, at least one of the output keywords 130 and 140 may be displayed together with caller and time information, which is information related to a call, as illustrated. As another example, as illustrated, the output keywords 130 and 140 generated by the artificial neural network extraction model may be displayed in the form of a keyword tag 150. The keyword tag 150 may maintain the original text of the output keyword 140 like the output keyword 140 corresponding to'place'. In addition, as shown, the output keyword 130 corresponding to the'date and time' may be processed and displayed as a keyword tag 150. For example, when the output keyword 130 is processed, the output keyword 130 may be input to the learned artificial neural network keyword processing model and the keyword tag 150 may be output.

According to an embodiment, the artificial intelligence assistant system may provide an interface 160 for searching the output text 120 generated by the artificial neural network voice-text transcription model. For example, after clicking on the search interface 160 of the artificial intelligence assistant system, the user may search for a related call by inputting a text to be searched.

2 is a block diagram illustrating an internal configuration of an artificial intelligence assistant system 200 according to an embodiment of the present disclosure. The artificial intelligence assistant system 200 may include a relay unit 210, a communication unit 220, a DB 230, a voice recognition unit 240, and a speech synthesis unit 250. In addition, the voice recognition unit 240 may include a voice storage module 242, a text transcription module 244, and a keyword extraction module 246.

The relay unit 210 of the artificial intelligence assistant system 200 may be configured to relay a call between a plurality of user terminals and receive voice data generated during the relayed call. According to an embodiment, when a call between a plurality of user terminals corresponds to a phone call through a public switched telephone network, the relay unit 210 calls the call through the public switched telephone network through a telephone call through the voice Internet protocol. Calls between a plurality of user terminals may be relayed so as to be performed. For example, the public switched telephone network may include a wired telephone network, a cellular network, and the like, and may include any network required for telephone calls between wireless terminals using a base station.

The relay unit 210 may receive voice data generated during a call between a plurality of user terminals. According to an embodiment, the relay unit 210 is the first voice data corresponding to the voice from the first user terminal (for example, caller) among the plurality of user terminals while a call between a plurality of user terminals is being performed. Can receive. Also, the relay unit 210 may receive second voice data corresponding to voice from a second user terminal (eg, a receiver) among a plurality of user terminals. The received voice data may be provided to the voice recognition unit 240 to perform voice-text transcription and further keyword extraction based on the voice data.

Accordingly, the user terminal can receive a plurality of phone numbers while using a single universal subscriber identification module (USIM) and use them during a phone call.

The communication unit 220 may be configured to allow an external device or system to communicate with the artificial intelligence assistant system. Here, the external device or system may provide web-based file sharing and synchronization services such as Google Spreadsheet, Google Drive, and Drop Box, but is not limited thereto. According to an embodiment, an external device or system may link information stored in the DB 230 of the artificial intelligence assistant system 200 with a service provided by it. For example, the communication unit 220 may provide a plurality of user terminals to a DB of voice data generated or received by the artificial intelligence assistant system of the present disclosure, output text, output keywords, keyword tags, summary text, call details, and text details. 230) may be configured to transmit and receive data to be shared, and the transmission/reception data is not limited thereto.

The DB 230 may store data generated, output, or received by the artificial intelligence assistant system of the present disclosure. In one embodiment, the DB 230 includes voice data generated during a call between a plurality of user terminals, an output text corresponding to the voice data, an output keyword extracted from the output text, a keyword tag, synthesized voice data, a call history, The text description and text content of the user terminal may be stored, but is not limited thereto. According to an embodiment, the DB 230 may be interlocked with data of an external service provided by an external device or system by the communication unit 220. In FIG. 2, the DB 230 is shown to be included in the artificial intelligence assistant system 200, but is not limited thereto, and is included in any external device accessible by wireless/wired communication by the artificial intelligence assistant system 200. I can.

The voice recognition unit 240 may store the voice data received from the relay unit 210 and generate text corresponding to the voice data through voice-text transcription. Furthermore, a keyword associated with a currency may be extracted from the generated text. In FIG. 2, the voice recognition unit 240 is shown to be included in the artificial intelligence assistant system, but is not limited thereto, and the voice recognition unit 240 may be provided to the user terminal.

The voice storage module 242 of the voice recognition unit 240 may store voice data received from the relay unit 210. In one embodiment, the voice storage module 242 may receive the caller's voice data and the callee's voice data from the relay unit 210, and store the caller's voice data and the receiver's voice data, respectively. have. For example, voice data generated during a call may be stored in the DB 230.

In one embodiment, the voice storage module 242 may store the time elapsed from the start time of the call of the caller and the callee of the call. In addition, the voice storage module 242 may remove a blank section in which the caller and the callee do not speak from the voice data and store it in association with time information elapsed from the call start time, but is not limited thereto.

The text transcription module 244 of the voice recognition unit 240 may be configured to input voice data into an artificial neural network voice-text transcription model and output text corresponding to the voice data. According to an embodiment, the text transcription module 244 may generate output data corresponding to the voice data in real time by inputting voice data received during a call into the artificial neural network voice-text transcription model. For example, the artificial neural network speech-to-text transcription model may be stored in the DB 230 and accessed by the text transcription module 244 upon learning and/or inference for speech-to-text transcription.

According to an embodiment, the text transcription module 244 may be configured to learn an artificial neural network speech-to-text transcription model using reference speech data and text corresponding to the reference speech data. In addition, voice data received during a call may be input to the learned voice-to-text transcription model, so that text corresponding to the voice data may be inferred or output. A configuration for learning or inferring such an artificial neural network voice-text transcription model will be described with reference to FIG. 13 below.

The keyword extraction module 246 of the speech recognition unit 240 may be configured to input at least some text of the output text into the artificial neural network keyword extraction model and output the keyword. According to an embodiment, at least some texts of output text corresponding to voice data received during a call may be input into an artificial neural network keyword extraction model to generate output keywords related to voice data. For example, the artificial neural network keyword extraction model may be stored in the DB 230, and may be accessed by the keyword extraction module 246 when learning and/or inferring for keyword extraction from text.

The speech synthesis unit 250 may be configured to input a given text into an artificial neural network text-speech synthesis model and output synthesized speech data corresponding to the text. According to an embodiment, the speech synthesis unit 250 may provide synthesized speech data corresponding to the text generated by the artificial intelligence service system to the user terminal and/or another user terminal in a call with the user terminal. For example, when a user terminal receives a missed call through the AI assistant system, the AI assistant system can transmit voice data corresponding to the text, such as a question about the reason for sending the call, to the missed call sender. And is not limited thereto.

In FIG. 2, the speech recognition unit 240 and/or the speech synthesis unit 250 are illustrated to be included in the artificial intelligence assistant system, but are not limited thereto, and the speech recognition unit 240 and/or the speech synthesis unit 250 May be stored and operated in any user terminal device capable of making a phone call through an artificial intelligence assistant system. Further, in FIG. 2, the relay unit 210, the voice recognition unit 240, and the speech synthesis unit 250 are shown as separate units, but are not limited thereto, and the relay unit 210 and the voice recognition unit 240 And any combination of the speech synthesis unit 250 may be configured as one unit. In addition, in FIG. 2, the voice storage module 242, the text transcription module 244, and the keyword extraction module 246 are configured as separate modules in the voice recognition unit 240, but are not limited thereto, and the voice storage module ( Any combination of 242), text transcription module 244, and keyword extraction module 246 may be configured as one module.

3 is a block diagram illustrating an artificial intelligence assistant system 200 for performing automatic voice-to-text (STT) transcription by connecting a public switched telephone network (PSTN) according to an embodiment of the present disclosure to a voice Internet protocol (VoIP). .

According to an embodiment, as shown in FIG. 3, the artificial intelligence assistant system 200 includes user terminals 310_1, 310_2, ..., 310_n and external user terminals 320_1, 320_2, ..., 320_n. Between Public Switched Telephone Network (PSTN) telephones can be converted to Voice Internet Protocol (VoIP) to connect telephones. In addition, the artificial intelligence assistant system 200 receives the voice data generated during a call between the user terminal (310_1, 310_2, ..., 310_n) and the external user terminal (320_1, 320_2, ..., 320_n), By inputting the received voice data into the artificial neural network voice-text transcription model included in the voice recognition unit 240, output text corresponding to the voice data may be generated in real time.

When the external user terminal (320_1, 320_2, ..., 320_n) transmits a call to the user terminal (310_1, 310_2, ..., 310_n) using a public switched telephone network (PSTN), the external user terminal (320_1, 320_2, ..., 320_n) may be connected to the communication company servers 330_1, 330_2, ..., 330_n through a public switched telephone network (PSTN). The communication service provider servers 330_1, 330_2, ..., 330_n using the public switched telephone network (PSTN) may be connected to the relay communication service provider server 340 contracted with the operator operating the artificial intelligence assistant system 200 of the present invention. . The relay communication company server 340 may be connected to the communication company servers 330_1, 330_2, ..., 330_n through a public switched telephone network (PSTN), and the relay communication company server 340 is an artificial intelligence assistant system 200 and voice Internet. It is connected through a protocol (VoIP) to transmit and receive data. Here, the relay communication company server 340 may convert transmission/reception data of a public switched telephone network (PSTN) into transmission/reception data of a voice Internet protocol (VoIP).

The artificial intelligence assistant system 200 may transmit and receive data by connecting the relay communication service provider server 340 and the user terminals 310_1, 310_2, ..., 310_n through a voice internet protocol (VoIP).

In one embodiment, the artificial intelligence assistant system 200 is an external user terminal (320_1, 320_2, ..., 320_n) passing through the communication company server (330_1, 330_2, ..., 330_n) and the relay communication company server 340 Can receive voice data. In addition, the artificial intelligence assistant system 200 may receive voice data transmitted from the user terminals 310_1, 310_2, ..., 310_n.

The artificial intelligence assistant system 200 receives first voice data corresponding to a voice from a first user terminal among a plurality of user terminals, and second voice data corresponding to a voice from a second user terminal among a plurality of user terminals. Can receive. In addition, the received voice data may be input into the artificial neural network voice-text transcription model to generate a first output text corresponding to the first voice data and a second output text corresponding to the second voice data. According to an embodiment, the voice recognition unit 240 of the artificial intelligence assistant system 200 includes user terminals 310_1, 310_2, ..., 310_n and external user terminals 320_1, 320_2, ..., 320_n. By inputting the voice data transmitted from each into the artificial neural network voice-text transcription model of the voice recognition unit 240, output text corresponding to the voice data may be generated in real time. That is, the artificial intelligence assistant system 200 separates and generates the output text corresponding to the voice data received from each of the caller and the receiver during a call, so that the voice recognition rate may be further improved.

FIG. 4 is a block diagram illustrating an artificial intelligence assistant system 200 for performing automatic voice-to-text (STT) transcription by connecting a public switched telephone network (PSTN) according to another embodiment of the present disclosure to a voice Internet protocol (VoIP). .

In one embodiment, as shown in Figure 4, the artificial intelligence assistant system 200 between the user terminals (310_1, 310_2, ..., 310_n) and external user terminals (320_1, 320_2, ..., 320_n) You can connect by converting your public switched telephone network (PSTN) phone to Voice Internet Protocol (VoIP). In addition, the artificial intelligence assistant system 200 receives voice data generated during a call between the user terminals 310_1, 310_2, ..., 320_n and the external user terminals 320_1, 320_2, ..., 320_n, By inputting the received voice data into the artificial neural network voice-text transcription model included in the voice recognition unit 240, output text corresponding to the voice data may be generated in real time. In this embodiment, the artificial intelligence assistant system 200 performs the same function of relaying and connecting to the public switched telephone network (PSTN) and voice Internet protocol (VoIP) of the relay communication service provider server 340 shown in FIG. 3. can do.

5 is a block diagram of an artificial intelligence assistant system 200 that performs automatic voice-to-text (STT) transcription in Internet Protocol (VoIP) according to an embodiment of the present disclosure.

As shown in FIG. 5, according to an embodiment, the artificial intelligence assistant system 200 relays a voice Internet protocol (VoIP) phone call between a plurality of user terminals 310_1, 310_2, ..., 310_n. I can. In addition, the artificial intelligence assistant system 200 receives voice data generated during a call between a plurality of user terminals 310_1, 310_2, ..., 310_n, and includes the received voice data in the voice recognition unit 240 The artificial neural network voice-text transcription model can be input, and output text corresponding to the voice data can be generated in real time, and the artificial intelligence assistant system 200 capable of performing the functions described above in Figs. It is natural for a person skilled in the art that a phone call between a plurality of user terminals 310_1, 310_2, ..., 310_n can also be relayed using the Internet Protocol (VoIP).

6 is a block diagram illustrating an artificial intelligence assistant system 200 that performs voice-to-text (STT) transcription in a user terminal according to another embodiment of the present disclosure.

As shown in FIG. 6, user terminals 310_1, 310_2, ..., 310_n of the artificial intelligence assistant system according to an embodiment may include voice recognition units 610_1, 610_2, 610_3. The voice recognition units 610_1, 610_2, 610_3 can also perform the same function as the function of generating the output text corresponding to the voice data of the voice recognition unit 240 described above in FIGS. 2 to 5 in real time. It is natural for the engineer of

Although not shown in FIG. 6, the user terminals 310_1, 310_2, ..., 310_n may include a speech synthesis unit (not shown). In another embodiment, the speech synthesis unit (not shown) may synthesize speech data based on text data generated by the user terminal 310_1, 310_2, ..., 310_3 or the artificial intelligence assistant system 200, , More specifically, it will be described later in FIG. 11.

7 is a flowchart illustrating a method 700 of performing speech-to-text (STT) transcription according to an embodiment of the present disclosure. The method 700 for performing voice-to-text (STT) transcription may be performed by the artificial intelligence assistant system 200 or the user terminals 310_1, 310_2, ..., 310_n. As shown, the method 700 for performing voice-to-text (STT) transcription may be initiated with a step S710 of receiving voice data generated during a call between a plurality of user terminals.

Then, in step S720, the received voice data is input to the artificial neural network voice-text transcription model, so that the output text corresponding to the voice data may be generated in real time. Finally, in step S730, the output text generated in real time may be stored.

8 is a flow chart illustrating a method 800 of connecting a public switched telephone network (PSTN) to a voice Internet protocol (VoIP) according to an embodiment of the present disclosure. The method 800 of connecting a public switched telephone network (PSTN) to a Voice Internet Protocol (VoIP) may be performed by the artificial intelligence assistant system 200. As shown, a method 800 of connecting a public switched telephone network (PSTN) to a Voice Internet Protocol (VoIP) may be initiated with a step (S810) of converting a wireless communication network telephone between a plurality of user terminals into an Internet telephone. have.

Then, in step S820, voice data generated during a call between a plurality of user terminals may be received.

Next, in step S830, the received voice data is input into the artificial neural network voice-text transcription model, and output text corresponding to the voice data may be generated in real time.

Finally, in step S840, the output text generated in real time may be stored.

9 illustrates a method 900 for displaying, tagging, and generating summary text for a keyword output based on an output text generated from a speech-text (STT) transcription according to an embodiment of the present disclosure This is a flow chart for A method 900 for displaying, tagging, and generating a summary text for the keyword output based on the output text generated from the speech-text (STT) transcription is an artificial intelligence assistant system 200 or a user terminal 310_1 , 310_2, ..., 310_n). As shown, a method 900 for displaying, tagging, and generating a summary text for a keyword output based on the output text generated from a speech-to-text (STT) transcription is a method 900 for generating the output text generated in real time. It may be started with the storing step (S910).

Then, in step S920, the output text generated in real time may be stored. In order to generate an output in step S920, the user's input may be received from the user terminals 310_1, 310_2, ..., 310_n. The artificial intelligence assistant system of the present disclosure may perform the steps of Case 1, Case 2, and Case 3 based on the received input.

In Case 1, when a request for'display output keyword' is received from a user as an input, an emphasis effect may be applied to the output keyword generated in step S930). Then, in step S935, the output keyword to which the highlighting effect is applied may be displayed.

In an embodiment, as described above with reference to FIG. 1, the artificial intelligence assistant system may apply an emphasis effect to the output keywords 130 and 140 and display an output keyword to which the emphasis effect is applied, but is not limited thereto. When a request for'tagging' is received as an input from a user in Case 2, at least one of the output keywords generated in step S940 may be displayed together with information related to a call. According to an embodiment, information related to a call may include arbitrary data transmitted and received during a call, a call record, whether a call is transmitted/received, a call partner name, a call start time and a call time, etc., but are not limited thereto.

A method of displaying information related to a call together with information related to a call in response to a'tagging' request according to an embodiment will be described later in more detail with reference to FIG. 10. In case 3, when a request for'display output keyword' is received from the user as an input, a summary text may be generated using at least one of the output keywords generated in step S950. The summary text generated in step S955 may be displayed together with information related to the call.

10 is a diagram illustrating an example of keyword tagging based on output text corresponding to voice data according to an embodiment of the present disclosure. According to an embodiment, the user terminals 310_1, 310_2, ..., 310_n may receive the keyword tag 1010 from the artificial intelligence assistant system 200. The keyword tag 1010 may be generated using at least some text of the output text corresponding to the voice data received during a call, extracted from the artificial intelligence assistant system 200. As illustrated, the keyword tag 1010 may be displayed together with information related to a currency associated with the output text. For example, as shown, a keyword tag 1010 is displayed along with information related to a call performed by the user terminal (e.g., call log, whether or not to send/receive a call, call party name, call start time, call time, etc.) Can be.

According to an embodiment, the keyword tag 1010 may be generated by including or processing some of the output keywords output by the artificial neural network keyword extraction model based on the output text generated by the artificial neural network voice-text transcription model. . For example, the keyword tag 1010 may include a core keyword inferred by an artificial neural network keyword extraction model from the contents of a call. As another example, the keyword tag 1010 may include an output keyword that is the most output among keywords extracted by an artificial neural network keyword extraction model among call contents.

In some embodiments, the keyword tag 1010 may output a related keyword, which is a word that is not in an actual call transcription record, based on some of the output keywords extracted by the artificial neural network keyword extraction model among the call contents. That is, some of the output keywords may be processed to generate the keyword tag 1010. For example, the content of a call made by the user terminal 310_1, 310_2, ..., 310_n is'Hello, Mr. Kim, is this a meeting at 3 PM, Gangnam, right?' In the case of,'meeting schedule confirmation' may be output as a related keyword. Here, in extracting a related keyword to generate the keyword tag 1010, a conventional natural language processing technique, a deep learning technique, and a machine learning technique may be used.

11 is an exemplary view illustrating a method of sharing voice and text data with other user terminals 310_1, 310_2, ..., 310_3 according to an embodiment of the present disclosure. According to an embodiment, the artificial intelligence assistant system 200 may share the phone list 1130 associated with the project with a plurality of user terminals 310_1, 310_2, ..., 310_n. In addition, the phone list 1130 related to the project may be stored in chronological order, but is not limited thereto.

In one embodiment, the user terminals 310_1, 310_2, ..., 310_n receive data stored in the DB 230 by the artificial intelligence assistant system 200, and the phone list 1130 associated with the project (for example, For example, the user terminals 310_1, 310_2, ..., 310_n may display and play back a phone call record, voice data, voice-to-text transcription data, and summary text, which are not directly performed.

For example, the user terminal 310_1 is related to a project to a plurality of other user terminals 310_2 and 310_n, such as a project sharing function of a conventional file sharing service (eg, Google Spreadsheet, Google Drive, Drop box, etc.). The phone list 1130 can be shared.

The user terminal 310_1 may modify the project name using the project name change window 1110.

In addition, the user terminal 310_1 may connect a plurality of other user terminals 310_2 and 310_n to the project using the project sharing window 1120.

In one embodiment, the phone list associated with the project is, for example, the user terminal 310_1, 310_2, ..., 310_n, the phone call list associated with the project, call voice data, voice-to-text transcription data, output keywords and summary Data stored in the DB 230 such as text may be included.

The phone list 1130 associated with the project will include some of the voice-to-text transcription data along with information related to the phone call (e.g., call destination, call start time, text record, etc.), as shown in FIG. Can be and is not limited thereto.

In another embodiment, the phone list 1130 associated with the project may include all or summary text of the voice-to-text transcription data, instead of some of the voice-to-text transcription data.

The summary text may be generated by inputting at least some text of the stored output text into the artificial neural network keyword extraction model by the keyword extraction module 246, and words not included in the speech-text transcription data may be added. I can. In generating the summary text, conventional natural language processing techniques, deep learning techniques, machine learning techniques, and the like may be used.

12 is an exemplary diagram for explaining automatic response and missed message management by the artificial intelligence assistant system 200 when receiving a missed call according to an embodiment of the present disclosure. According to an embodiment, the artificial intelligence assistant system 200 or the user terminal 310_1, 310_2, ..., 310_n is a case in which the user of the user terminal 310_1, 310_2, ..., 310_n cannot answer a call , Can provide interactive automatic response service.

By the voice synthesis unit 250 included in the artificial intelligence assistant system 200 or the voice synthesis unit (not shown) included in the user terminals 310_1, 310_2, ..., 310_n, a call is transmitted when the user is absent. Voice data synthesized based on the missed message created by the user or the missed message generated by the artificial intelligence assistant system 200 may be transmitted to a sender.

In another embodiment, voice data pre-recorded by the artificial intelligence assistant system 200 may be transmitted to the caller who sent the call when the user is absent.

In some embodiments, voice data previously recorded by the user may be transmitted to the caller who sent the call when the user is absent, but is not limited thereto.

The voice recognition unit 240 included in the artificial intelligence assistant system 200 or the voice recognition units 610_1, 610_2, 610_3 included in the user terminals 310_1, 310_2, ..., 310_n, the voice transmitted to the caller By receiving the caller's response corresponding to the data, voice-to-text transcription can be performed. The keyword extraction module 246 of the speech recognition unit 240 may receive the voice-text transcribed data and output an output keyword, tag, or summary text.

As illustrated in FIG. 12, the artificial intelligence assistant system 200 may transmit a summary text 1210 corresponding to the caller's response to the user terminals 310_1, 310_2, ..., 310_n.

As shown in FIG. 12, through the call list window 1220 of the user terminal 310_1, 310_2, ..., 310_n, a call or missed call record can be checked. In addition, it is possible to display and reproduce voice data or voice-to-text transcription data of a call or missed call and information related to the call.

As shown in FIG. 12, through the text list window 1230 of the user terminal 310_1, 310_2, ..., 310_n, full text of received and sent texts may be displayed.

In another embodiment, incoming and outgoing characters may be input to the keyword extraction module 246 and output keywords or tags or summary text corresponding to incoming and outgoing characters may be output in the character list window 1230.

As shown in FIG. 12, through the pipeline 1240, call and text details associated with a caller may be displayed in chronological order, but is not limited thereto.

13 is an exemplary diagram illustrating an artificial neural network model 1300 according to an embodiment of the present disclosure. The artificial neural network model 1300 is a statistical learning algorithm implemented based on the structure of a biological neural network or a structure that executes the algorithm in machine learning technology and cognitive science. According to an embodiment, in the artificial neural network model 1300, nodes, which are artificial neurons that form a network by combining synapses as in a biological neural network, repeatedly adjust the weight of the synapse, By learning to reduce an error between the output and the inferred output, it is possible to represent a machine learning model having problem solving ability. For example, the artificial neural network model 1300 may include an arbitrary probability model, a neural network model, and the like used in artificial intelligence learning methods such as machine learning and deep learning.

According to an embodiment, the artificial neural network model 1300 may include an artificial neural network speech-to-text transcription model configured to input training speech data and/or speech data features extracted from the learning speech data to output output text. . For example, such an artificial neural network speech-to-text transcription model may be learned for each user, and a speech recognition technology optimized for each user may be provided. That is, the artificial neural network voice-text transcription model is generated for each user terminal or user, and can be used for learning or inference for speech-text transcription for each user terminal or for each user. In another embodiment, the artificial neural network model 1300 may include an artificial neural network keyword extraction model configured to receive an output text and output an output keyword. In another embodiment, the artificial neural network model 1300 may include an artificial neural network text-speech synthesis model configured to receive input text and output synthesized speech data.

The artificial neural network model 1300 is implemented as a multilayer perceptron (MLP) composed of multilayer nodes and connections between them. The artificial neural network model 1300 according to the present embodiment may be implemented using one of various artificial neural network model structures including MLP. As shown in FIG. 13, the artificial neural network model 1300 includes an input layer 1320 that receives an input signal or data 1310 from the outside, and an output layer that outputs an output signal or data 1350 corresponding to the input data. (1340), which is located between the input layer 1320 and the output layer 1340, receives a signal from the input layer 1320, extracts a characteristic, and transmits it to the output layer 1340 (where n is a positive integer). It consists of hidden layers 1330_1 to 1330_n. Here, the output layer 1340 receives signals from the hidden layers 1330_1 to 1330_n and outputs them to the outside.

In the learning method of the artificial neural network model 1300, a supervised learning method that learns to be optimized to solve a problem by inputting a teacher signal (correct answer), and an unsupervised learning method that does not require a teacher signal ) There is a way. The text transcription module 244 analyzes the input speech data using supervised learning in order to output text based on the learning speech data, so that the output text corresponding to the input speech data can be inferred. The artificial neural network model 1300 may be trained. The artificial neural network model 1300 learned in this way may be stored in the DB 230 and may output text in response to an input of voice data received from the communication unit 220 and/or an input device. The keyword extraction module 246 analyzes the input text using supervised learning to output keywords based on the learning text data, and an artificial neural network so that the output keywords corresponding to the input text can be inferred. The model 1300 may be trained. The artificial neural network model 1300 learned in this way may be stored in the DB 230 and may output a keyword in response to an input of text received from the communication unit 220 and/or the input device. The speech synthesis unit 250 analyzes the input text using supervised learning in order to output speech data based on the learning text data, so that the output speech data corresponding to the input text can be inferred. The artificial neural network model 1300 may be trained. The artificial neural network model 1300 learned in this way may be stored in the DB 230 and may output voice data in response to an input of text received from the communication unit 220 and/or an input device.

According to an embodiment, as illustrated in FIG. 13, the input variable of the artificial neural network model 1300 capable of extracting the output text may be training voice data including at least part of a conversation. For example, an input variable input to the input layer 1320 of the artificial neural network model 1300 may be a speech feature vector 1310 in which training speech data is composed of one vector data element. In response to input of learning speech data including at least a portion of the conversation, an output variable output from the output layer 1340 of the artificial neural network model 1300 may be a vector 1350 representing the output text. In addition, the output layer 1340 of the artificial neural network model 1300 may be configured to output a vector indicating reliability and/or accuracy of the output text. For example, a vector representing confidence or accuracy for such an output can be interpreted or displayed as a score. In the present disclosure, the output variable of the artificial neural network model 1300 is not limited to the type described above, and may include arbitrary information/data representing the output text.

In this way, a plurality of output variables corresponding to a plurality of input variables are matched to the input layer 1320 and the output layer 1340 of the artificial neural network model 1300, respectively, and the input layer 1320, the hidden layers 1330_1 to 1330_n, and the output layer By adjusting the synaptic values between nodes included in 1340, it may be learned so that a correct output corresponding to a specific input can be extracted. Through this learning process, nodes of the artificial neural network model 1300 can identify the characteristics hidden in the input variables of the artificial neural network model 1300 and reduce the error between the output variable calculated based on the input variable and the target output. You can adjust the synaptic values (or weights) between them. In response to the input voice data, information on the output text corresponding to the input voice data may be output using the artificial neural network model 1300 thus learned.

According to another embodiment, an input variable of the artificial neural network model 1300 capable of generating an output keyword may include a vector representing information corresponding to the input text. In response to these input variables, the output layer 1340 of the artificial neural network model 1300 may be a vector representing an output keyword. Here, a weight value between nodes included in the input layer 1320, the hidden layers 1330_1 to 1330_n, and the output layer 1340 may be adjusted so that a correct output corresponding to a specific input may be extracted. The artificial neural network model 1300 capable of generating such an output keyword may be stored in the DB 230. Using the artificial neural network model 1300 learned in this way, information on an output keyword may be output in response to information on the input text.

According to another embodiment, the input variable of the artificial neural network model 1300 capable of generating voice data may include a vector representing information on the input text. In response to these input variables, the output layer 1340 of the artificial neural network model 1300 may be a vector representing voice data. Here, a weight value between nodes included in the input layer 1320, the hidden layers 1330_1 to 1330_n, and the output layer 1340 may be adjusted so that a correct output corresponding to a specific input may be extracted. The artificial neural network model 1300 capable of generating such voice data may be stored in the DB 230. In response to information on input text, information on voice data may be output using the artificial neural network model 1300 learned in this way.

The above-described method for managing the contents of a call may be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium is distributed over a computer system connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described embodiments can be easily inferred by programmers in the technical field to which the present invention belongs.

The method, operation, or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those of skill in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or combinations of both. To clearly illustrate this interchange of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs). ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure , Computer, or a combination thereof.

Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may include a general purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or It may be implemented or performed in any combination of those designed to perform the functions described in. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in connection with the DSP core, or any other configuration.

In firmware and/or software implementation, the techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on a computer-readable medium such as programmable read-only memory, erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, etc. It can also be implemented as stored instructions. The instructions may be executable by one or more processors, and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

When implemented in software, the techniques may be stored on a computer-readable medium as one or more instructions or code or transmitted through a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of non-limiting example, such computer-readable medium may contain RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or the desired program code in the form of instructions or data structures. It may include any other medium that may be used for transfer or storage to and accessible by a computer. Also, any connection is properly termed a computer-readable medium.

For example, if the software is transmitted from a website, server, or other remote source using wireless technologies such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared, wireless, and microwave, coaxial cable , Fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, wireless, and microwave are included within the definition of the medium. Disks and discs as used herein include CDs, laser disks, optical disks, digital versatile discs (DVDs), floppy disks, and Blu-ray disks, where disks are usually magnetic It reproduces data optically, whereas discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other known type of storage medium. An exemplary storage medium may be coupled to a processor such that the processor can read information from or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may also reside within the ASIC. The ASIC may exist in the user terminal. Alternatively, the processor and storage medium may exist as separate components in the user terminal.

Although the above-described embodiments have been described as utilizing aspects of the currently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto, and may be implemented in connection with any computing environment such as a network or a distributed computing environment. . Furthermore, aspects of the subject matter in this disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include PCs, network servers, and portable devices.

In the present specification, the present disclosure has been described in connection with some embodiments, but various modifications and changes may be made without departing from the scope of the present disclosure as understood by those of ordinary skill in the art to which the present disclosure belongs. In addition, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

200: server 210: relay
220: communication unit 230: DB
240: voice recognition unit 242: voice storage module
244: text transcription module 246: keyword extraction module
250: speech synthesis unit
310_1, 310_2, ..., 310_n: user terminal
320_1, 320_2, ..., 320_n: external user terminal
330_1, 330_2, ..., 330_n: carrier server
340: relay carrier server

Claims (10)

In how to manage the call details,
When a call between a plurality of user terminals corresponds to a call through a public switched telephone network (PSTN), a call through the public switched telephone network is performed through Voice over Internet Protocol (VoIP). Converting to a currency;
First voice data corresponding to the voice from the caller terminal for the call and second voice data corresponding to the voice from the receiver terminal for the call, generated during a call through the voice Internet protocol between the plurality of user terminals Receiving;
By inputting the received first voice data and second voice data into an artificial neural network speech-to-text transcription model, the first output text and the second voice data corresponding to the first voice data Generating a second output text corresponding to the in real time; And
Storing the first output text and the second output text generated in real time
Containing, how to manage the content of the call.
The method of claim 1,
And generating an output keyword by inputting at least some text of the stored output text into an artificial neural network keyword extraction model.
The method of claim 2,
Applying an emphasis effect to the generated output keyword; And
Displaying an output keyword to which the highlighting effect is applied
How to manage the call content further comprising.
The method of claim 2,
Generating a keyword tag using at least some of the generated output keywords;
Further comprising displaying the generated keyword tag together with information related to the currency.
The method of claim 2,
Generating a summary text using at least some of the generated output keywords; And
And displaying the generated summary text together with information associated with the call.
delete A computer program stored in a computer-readable recording medium for executing the method for managing call contents according to any one of claims 1 to 5 on a computer.
In the artificial intelligence assistant system,
A relay unit configured to relay a call between a plurality of user terminals and receive voice data generated during the relayed call; And
It is configured to input the received speech data into an artificial neural network speech-to-text transcription model to generate an output text corresponding to the speech data in real time, and to store the output text generated in real time. Including a voice recognition unit,
The relay unit,
When a call between the plurality of user terminals corresponds to a call through a public switched telephone network (PSTN), a call through the public switched telephone network is performed using Voice over Internet Protocol (VoIP). Relaying a call between the plurality of user terminals to switch to a call through,
Receiving first voice data corresponding to the voice from the caller terminal for the call among the plurality of user terminals,
Further configured to receive second voice data corresponding to a voice from a receiver terminal for the call among the plurality of user terminals,
Artificial intelligence assistant system.
The method of claim 8,
The speech recognition unit is further configured to generate an output keyword by inputting at least some text of the stored output text into an artificial neural network keyword extraction model.
delete

Images