JP2026000240A - Minutes creation support device and program - Google Patents

Abstract

A minutes-creation support device and program are provided that can create meeting minutes in an optimal manner.
[Solution] A minutes creation support device comprising: an acquisition unit that acquires text data indicating the content of speeches spoken at a meeting; a processing unit that acquires a sentence summarizing the speeches at the meeting by inputting a prompt containing different parts for each type of agenda item to request the creation of a sentence summarizing the speeches at the meeting and information based on the text data into a large-scale language model; and an output unit that outputs minutes of the meeting based on the acquired sentence.
[Selected Figure] Figure 1

Description

Application for application of Article 30, Paragraph 2 of the Patent Act filed (1) Publication date: July 3, 2023 Website address: https://www.youtube.com/watch?v=wqKqBDZvNKY&t=4s (2) Publication date: July 12, 2023 Website address: https://michibiku.co.jp/news/si10q4t9jts/ (3) Publication date: July 12, 2023 Website address: https://www.nikkei.com/article/DGXZRSP658954_R10C23A7000000/

The present invention relates to a minutes creation support device and program.

Research and practical applications have been ongoing into automatically creating minutes from text data showing the content of meetings. For example, Patent Document 1 describes extracting key points using keywords to create a summary of minutes. It also describes dividing text data into sections corresponding to a type selected from multiple section types and generating summary information for each divided data. Patent Document 2 describes storing key terms related to each of multiple agenda items and creating minutes by segmenting the text data based on the appearance of the key terms. Patent Document 3 describes creating minutes by detecting changes in agenda items based on information attached to the meeting material data. Patent Document 4 describes setting agenda items such as reports and resolutions, storing text data in association with the agenda items, and displaying meeting information when a user selects an agenda item.

Japanese Patent Application Laid-Open No. 2019-139572 JP 2023-178758 A Patent No. 6340926 Japanese Patent Application Laid-Open No. 2023-001849

The technologies described in Patent Documents 1 to 3 detect changes in meeting agendas solely based on information such as keywords and additional information. Furthermore, the technology described in Patent Document 4 simply switches display data in response to the selection of an agenda. With conventional technologies, it has sometimes been difficult to create meeting minutes appropriately.

The present invention was made in consideration of these circumstances, and one of its objectives is to provide a minutes-creation support device and program that can efficiently create meeting minutes.

One aspect of the present invention is a minutes-creation support device that includes an acquisition unit that acquires text data indicating the content of utterances made at a meeting; a processing unit that acquires a sentence summarizing the utterances made at the meeting by inputting a prompt that includes a different part for each type of topic and information based on the text data into a large-scale language model to request the creation of a sentence summarizing the utterances made at the meeting; and an output unit that outputs minutes of the meeting based on the acquired sentence.

Another aspect of the present invention is a minutes-creation support device that includes an acquisition unit that acquires text data indicating the content of utterances at a meeting; a processing unit that acquires a sentence summarizing the utterances at the meeting by inputting, into a large-scale language model, a prompt that differs for each type of minutes and indicates the degree to which the content of the utterances at the meeting should be summarized, and information based on the text data, in order to request the creation of a sentence summarizing the utterances at the meeting; and an output unit that outputs the minutes of the meeting based on the acquired sentence.

Another aspect of the present invention is a program for causing a computer to execute the following processes: acquiring text data indicating the content of speeches spoken at a meeting; acquiring a sentence summarizing the speeches at the meeting by inputting a prompt containing a different part for each type of agenda item and information based on the text data into a large-scale language model to request the creation of a sentence summarizing the speeches at the meeting; and outputting minutes of the meeting based on the acquired sentence.

According to one aspect of the present invention, meeting minutes can be created efficiently.

1 is a diagram showing an example of the configuration of a minutes-taking assistance device 100 according to the present invention and a surrounding environment thereof; 10 is a flowchart showing an example of a flow of processes executed by each unit of the minutes-taking support device 100. FIG. 10 is a diagram showing an example of the contents of conference information 152. FIG. 10 is a diagram showing an example of an interface screen IM1 for receiving input from a speaker. FIG. 10 is a diagram conceptually illustrating a process for creating a prompt. 3 is a flowchart showing a detailed example of the process of S10 in FIG. 2; FIG. 10 is a diagram illustrating an example of the configuration of a minutes-taking assistance device 100A according to a first modification and a surrounding environment. FIG. 10 is a diagram illustrating an example of the configuration of a minutes-taking assistance device 100B according to a second modification and a surrounding environment.

Hereinafter, with reference to the drawings, an embodiment of the minutes-taking support device and program of the present invention will be described. The minutes-taking support device is a device that supports, for example, meetings held at a company, from the holding of the meetings to the creation of the minutes. In particular, the minutes-taking support device can effectively support the automatic creation of minutes of board meetings (including some user operations). In the following description, the minutes-taking support device is assumed to provide cloud services via a network, but the minutes-taking support device may also be an on-premise device installed at a client's facility.

[composition]
FIG. 1 shows an example of the configuration and surrounding environment of a minutes-creation support device 100 according to the present invention. The minutes-creation support device 100 communicates with a terminal device 10 and an LLM (Large Language Model) server 200 via a network. The terminal device 10 is a terminal device of a user who wishes to use the services provided by the minutes-creation support device 100, and is a computer device such as a personal computer, tablet terminal, or smartphone. A program such as a browser or a dedicated application program runs on the terminal device 10 to receive the services of the minutes-creation support device 100. The network includes the Internet, a cellular network, a dedicated line, a WAN (Wide Area Network), a LAN (Local Area Network), and the like.

The minutes creation support device 100 includes, for example, an IF (interface) unit 110, a processing unit 120, and a storage unit 150. The components other than the storage unit 150 are realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented using a large-scale integration (LSI), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).
The program may be realized by hardware (including circuitry) such as a Gate Array, a Graphics Processing Unit (GPU), or a System On Chip (SOC), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as a Hard Disk Drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device.

The storage unit 150 is a RAM (Random Access Memory), HDD, flash memory, or the like. Data and information such as meeting information 152, transcript text 154, and prompt format 156 are stored in the storage unit 150. The storage unit 150 does not necessarily have to be attached to the minutes-taking support device 100, but may also be attached to an external server. The external server is not limited to a cloud server, and may also be operated by a business operator that uses the minutes-taking support device 100.

The IF unit 110 has, for example, the functionality of a web server and provides web pages for providing the functions of the minutes creation support device 100 to the terminal device 10. The IF unit 110 includes, for example, an acquisition unit 112 and an output unit 114.

The acquisition unit 112 acquires text data indicating the content of the meeting speech. The processing unit 120 acquires a sentence summarizing the meeting speech for each type of agenda by inputting a prompt containing different parts for each type of agenda and information (manuscript) based on the text data into a large-scale language model (hereinafter referred to as LLM) provided by the LLM server 200 to request the creation of a sentence summarizing the meeting speech (the summary portion of the minutes). The types of agenda include, for example, items to be reported, items to be resolved, and items to be discussed. The output unit 114 outputs (displays) the minutes of the meeting to the terminal device 10 based on the acquired sentence. The functions of these functional units are explained below using a flowchart.

[Process flow]
FIG. 2 is a flowchart illustrating an example of the flow of processing executed by each unit of the minutes-creation support device 100. First, the IF unit 110 sets conference information based on information acquired from the terminal device 10 and stores it in the storage unit 150 as conference information 152 (S1). FIG. 3 is a diagram illustrating an example of the contents of the conference information 152. For example, the conference information 152 is information in which an automatically assigned conference ID is associated with information such as the conference title, attendees, and agenda items (reports, resolutions, and discussion items). Depending on the conference, some reports, resolutions, and discussion items may not be present. This information is input by the user of the terminal device 10 into the terminal device 10 and transmitted to the IF unit 110. Information about the minutes created as described below is added to the conference information 152, allowing the minutes to be later searched and obtained using the conference ID or conference title.

Returning to the explanation of Figure 2, the IF unit 110 sends a conference invitation notice to the attendees' terminal devices 10 based on the conference information 152 and attendee information (S2). The attendees' communication identification information is stored in the memory unit 150 in advance, for example, when the minutes creation support device 100 service is introduced.

After the meeting has begun (or in real time as the meeting progresses), the acquisition unit 112 acquires audio data of the meeting (S3). The audio data may be uploaded from the terminal device 10, or from a recording device to the minutes-creation support device 100.

The acquisition unit 112 performs text conversion processing on the speech data to convert it into text data for each speaker (S4). The text conversion processing includes, for example, speech recognition processing and morphological analysis processing. The text conversion processing may be performed by the minutes creation support device 100, or may be performed by requesting an external device.

Next, the acquisition unit 112 outputs the results of the text conversion process to the terminal device 10 and accepts input of information indicating which attendee made the remark (i.e., the speaker or speaker) for each utterance (S5). As a result, the acquisition unit 112 associates the utterance with the speaker, and stores the associated information in the storage unit 150 as transcribed text 154.

Figure 4 shows an example of an interface screen IM1 for accepting speaker input. Interface screen IM1 displays speaker (speaker) and speech content (utterance) in association with speech time (speech time). When icon A1 in the speaker section is operated, a list of attendees identified from the conference information 152 is presented as speaker candidates, allowing the user to select a speaker. After this operation, when a confirmation operation is performed, speaker information is acquired by the acquisition unit 112. Note that instead of having the user of the terminal device 10 input speaker information, the minutes-taking support device 100 may automatically recognize speakers by analyzing audio data. In this case, the minutes-taking support device 100 may present speaker candidates to the terminal device 10 and obtain approval from the user. When speaker recognition is performed automatically, data input by the user of the terminal device 10 several times may be used as correct answer data.

Next, the acquisition unit 112 accepts input of the type of minutes from the user of the terminal device 10 (S6). Minutes in this invention do not include everything that was said at the meeting, but are summarized to include important points. The type of minutes, for example, represents the degree of summarization (granularity) in stages. For example, three types of minutes are provided: concise (highest degree of summarization), leaving important questions and answers, and leaving many questions and answers (lowest degree of summarization), and the user of the terminal device 10 selects the type of minutes from these.

Next, the processing unit 120 creates a prompt to be entered into the LLM (S7). The processing unit 120 creates a prompt with different answer form sections and/or instruction information for each type of topic (S7).

Figure 5 is a conceptual diagram illustrating the process of creating a prompt. The processing unit 120 reads a prompt format corresponding to the type of minutes selected by the user from the prompt format 156, and creates a prompt by appropriately embedding the agenda information included in the conference information 152 into the prompt format. The prompt format 156 includes three prompt formats, one for each type of minutes, and each prompt format has a pre-defined answer form section and instruction information. The answer form section and instruction information have content that differs, at least in part, depending on the type of agenda. In this way, the processing unit 120 creates a prompt that includes different sections for each type of agenda. Note that other information, in addition to agenda information, may be quoted from the conference information 152.

Returning to the explanation of Figure 2, the processing unit 120 first inputs a prompt for creating a summary of the minutes and the transcribed text (manuscript) into the LLM (S8). As a result, the processing unit 120 obtains, for example, a sentence summarizing the meeting speech for each type of agenda. The sentence summarizing the meeting speech for each type of agenda may be a sentence divided into blocks for each type of agenda and topic and labeled with a label indicating the type of agenda and topic, or a sentence in which headings or meta information indicate which part corresponds to which type of agenda and topic.

Next, the processing unit 120 determines whether the type of minutes is "concise" (S9). If the type of minutes is "concise," the output unit 114 edits the text obtained from the LLM (S30) and outputs it to the terminal device 10 (S31).

If the type of minutes is not "concise" but "leaves important questions" or "leaves many questions," the processing unit 120 inputs a prompt for extracting questions and the transcribed text (manuscript) into the LLM (S10).

Figure 6 is a flowchart showing a detailed example of the processing of S10 in Figure 2. First, the processing unit 120 divides the transcribed text into chunks of a size that can be input into the LLM (S11). Next, the processing unit 120 repeatedly executes the processing of S12 to S17 for each agenda item of the meeting (selecting each agenda item one by one and executing the processing for each). First, the processing unit 120 inputs a prompt for extracting questions and answers corresponding to the agenda item into the LLM (S12). Next, the processing unit 120 selects one chunk in chronological order (S13) and determines whether the chunk contains content related to the agenda item (S14). If the chunk does not contain content related to the agenda item, the processing returns to S13. If the chunk contains content related to the agenda item, the processing unit 120 determines whether the chunk contains the end of the agenda item (S15). If the end of the agenda item is not included in the chunk, the processing unit 120 inputs the chunk into the LLM, stores the answer (extracted question) in the intermediate table (S16), and returns to S13. If the end of the agenda item is included in the chunk, the processing unit 120 inputs the chunk into the LLM, and stores the answer together with previous answers related to the agenda item in the intermediate table (S17). In this way, when requesting the LLM to extract the content of the question and answer, the processing unit 120 identifies the section of the agenda item in the transcribed text (start and end points) and inputs this into the LLM.

The processing unit 120 performs the processes of S14 to S17 by determining, for example, whether a keyword corresponding to the agenda item is included, and whether a keyword such as "that's all" or "end," which indicates the end of the agenda item, is included. Note that the processing unit 120 may also perform the processes of S14 to S17 by separately requesting the LLM. Also, in the process of S8 in Figure 2, the transcribed text is divided into chunks and input into the LLM. At this time, prompts may be input into the LLM sequentially for each agenda item, as in the process of Figure 6.

When the processing unit 120 completes the processing of S8 to S10, it adds the type of agenda, summary, and corresponding questions and answers for each agenda item to the conference information 152, associating them with the conference ID. The relationship between the minutes and the agenda items may be determined from the information added by the LLM, or may be determined from the information on the agenda items that was the basis when the processing unit 120 requested processing from the LLM.

[Example of prompt]
An example of a prompt is described below. The prompt input to the LLM includes, as output format instruction information, instruction information such as "Exclude questions and answers from the output" if the type of minutes is "concise," and instruction information such as "Write the main explanation, followed by questions and answers (optional)" if the type of minutes is "Leave important questions and answers" or "Leave most of the questions and answers." Furthermore, instruction information for questions and answers includes instruction information such as "Rewrite the questions and answers and conversations between speakers in simple sentences without citing them" if the type of minutes is "Leave important questions and answers," and instruction information such as "Rewrite the questions and answers and conversations between speakers in the form of a detailed report" if the type of minutes is "Leave most of the questions and answers."

Below is an example of a prompt that may be created when the minutes type is "Leave important questions open." For minutes types of "Concise" or "Leave many questions open," portions of the prompt may be removed, added, or modified as appropriate to adjust the level of summary. Prompts may include a declaration of the meeting agenda, a response form section like those in the agenda section, and other instructional information, which may vary by agenda topic and may also vary by minutes type.

##################"Leave important questions"################
System
(From now on, references to "Q&A" should be considered to include both Q&A and conversation between speakers.)
You are a worker who rewrites the concise minutes of business meetings.
[Output format]
・Write the main explanation, followed by a Q&A (optional).
・Please add any questions or answers written in the manuscript.
・Create a business report written in formal Japanese.
- Use the passive voice when reporting something.
User
The manuscript is given below. The manuscript is divided into chunks, with each chunk continuing from the previous one.
[ ]
Meeting agenda:
[{"Reporting items", "Monthly financial results report for July 2022 to September 2022"},
{"Report items", "Monthly financial results report for October and November 2022"},
{"Report items", "Monthly financial report for December 2022 to February 2023"},
{"Resolution items", "Item 1: Fundraising (bank loans)"},
{"Resolution items", "Agenda item 2: Fundraising (bank loans)"},
{"Resolutions", "Agenda item 3: Establishment of work rules"}.
Format: [{type, topic}, {}...]
(Agenda items have one of three types: reports, resolutions, and discussion items.)
Main instructions: {Create and print minutes.

Items to report: {
Please use the following format:
"xx reported about yy."
Resolutions:
- Please report the resolutions, including whether they were approved or rejected, as follows:
Please use the following format:
"xx has resolved on yy."
Discussion items: {
Please use the following format:
"xx discussed yy."
Q&A:
• Rewrite the questions and answers and the dialogue between the speakers.
・Please write the content of the section of the manuscript that contains the question.
Q&A (include brackets) format:
¥n¥n (Opinion, summary of Q&A) ¥n (Speaker) Question 1 ¥n (Speaker) Answer 1 ¥n¥n (Speaker) Question 2... }
Please output in the same format as below.
{{"Monthly financial report for July 2022 to September 2022":"String: Main explanation (+Q&A)"},
{"Regarding the monthly financial results report for October and November 2022":""},
{"Monthly financial report for December 2022 to February 2023":""},
{"Item 1: Fundraising (Bank Loans)":""}
{"Agenda item 2: Fundraising (bank loans)":""},
{"Agenda item 3: Establishment of work rules":""}}
################################################

[Modification]
In the above description, the processing unit 120 inputs a so-called zero-shot prompt into the LLM. However, a few-shot prompt that includes past examples in the prompt may also be input into the LLM. In this case, the processing unit 120 may prepare sample data for each type of minutes and each type of agenda, created manually or using the LLM, and include the sample data in the prompt. Figure 7 shows an example of the configuration and surrounding environment of a minutes-creation support device 100A according to Variation 1. The memory unit 150 of the minutes-creation support device 100A stores minutes-creation sample data 158 by type. The processing unit 120 inputs the above-described prompt with the addition of "send minutes sample data" into the LLM, and reads the minutes sample data corresponding to the types of minutes and agenda from the memory unit 150 and inputs it into the LLM before or after the transcription text. In this way, the minutes-taking support device 100A can obtain a more appropriate response from the LLM.

In addition, instead of including sample data in the prompt, the processing unit 120 may include data associating the transcribed text with the summary of the created minutes in the prompt. Figure 8 is a diagram showing an example of the configuration and surrounding environment of the minutes creation support device 100B relating to variant example 2. The memory unit 150 of the minutes creation support device 100B stores manuscript/minutes correspondence data 160. The manuscript/minutes correspondence data 160 is, for example, data that associates the transcribed text divided by topic when minutes were created in the past, the type of minutes, and the summary of the minutes divided by topic. The summary of the minutes in the manuscript/minutes correspondence data 160 may be a summary revised by the user in response to a response from the LLM. The processing unit 120 extracts past transcribed text similar to the newly input transcribed text from the manuscript/minutes correspondence data 160, and includes the summary of the minutes divided by the corresponding topic in the prompt and inputs it to the LLM. For example, the processing unit 120 determines the similarity between two manuscripts by calculating the distance between distributed representation vectors that represent keywords contained in the two manuscripts. Alternatively, the processing unit 120 may use a thesaurus or the like to determine the similarity between manuscripts based on the presence of the same or similar keywords. In this way, the minutes-creation support device 100B can create minutes that better match the user's image.

In addition, the processing unit 120 may perform the above processing using an LLM fine-tuned by the minutes creation support device 100, rather than a general-purpose LLM. In this case, the processing unit 120 creates minutes summaries several times using the LLM, and then fine-tunes the LLM based on user feedback on the summaries. User feedback may be, for example, information indicating the user's level of satisfaction with the minutes summary entered by the user, or the degree of revisions made by the user to the minutes summary (it is presumed that the more revisions made, the lower the level of satisfaction). In this case, the processing unit 120 may perform fine-tuning separately for each type of minutes and each type of agenda, so that a separate LLM operates for each type of minutes and each type of agenda.

According to the embodiment described above, meeting minutes can be created efficiently by including an acquisition unit 112 that acquires text data indicating the content of the meeting speech, a processing unit 120 that acquires a sentence summarizing the meeting speech by inputting a prompt containing different parts for each type of topic and information based on the text data into a large-scale language model to request the creation of a sentence summarizing the meeting speech, and an output unit 114 that outputs the meeting minutes based on the acquired sentence.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is in no way limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

10 Terminal device 100, 100A, 100B Meeting minutes creation support device 110 IF unit 112 Acquisition unit 114 Output unit 120 Processing unit 150 Storage unit 152 Meeting information 154 Transcription text 156 Prompt format 158 Meeting minutes sample data by type 160 Manuscript/minutes corresponding data

Claims (14)

an acquisition unit that acquires text data indicating the contents of speeches made in a conference;
a processing unit that acquires a sentence summarizing the speech of the meeting by inputting a prompt including a portion that differs depending on the type of agenda and information based on the text data into a large-scale language model, the prompt being used to request the creation of a sentence summarizing the speech of the meeting;
an output unit that outputs minutes of the meeting based on the acquired sentences;
A minutes creation support device comprising: the processing unit acquires a sentence summarizing the speeches of the meeting for each type of agenda.
2. The minutes creation support device according to claim 1. The types of agenda items include reports, resolutions, and discussions.
3. The minutes creation support device according to claim 1 or 2. the processing unit inputs the prompt, which has a different answer form portion for each type of topic, into the large-scale language model;
3. The minutes creation support device according to claim 1 or 2. the processing unit inputs the prompt, which has different instruction information for each type of topic, into the large-scale language model;
3. The minutes creation support device according to claim 1 or 2. the processing unit inputs the prompt, which differs for each type of the minutes, into the large-scale language model;
2. The minutes creation support device according to claim 1. The type of minutes indicates the degree to which the contents of the speeches made in the meeting are summarized.
7. The minutes creation support device according to claim 6. The acquisition unit accepts a designation of the type of the minutes by a user.
8. The minutes creation support device according to claim 6 or 7. the processing unit requests the large-scale language model to extract the content of questions and answers that took place in the meeting, and at that time, identifies a section of the agenda in the text data and inputs the section to the large-scale language model.
2. The minutes creation support device according to claim 1. the processing unit reads sample data for the summary results for each type of agenda from a storage unit, includes the sample data in the prompt, and inputs the sample data into the large-scale language model;
2. The minutes creation support device according to claim 1. the processing unit refers to data associating the text data with the minutes of the meeting when processing was performed in the past, and inputs the minutes of the meeting corresponding to the past text data similar to the current text data into the large-scale language model by including the minutes of the meeting in the prompt.
2. The minutes creation support device according to claim 1. the large-scale language model can be fine-tuned by the minutes-taking assistance device;
the processing unit fine-tunes the LLM for each type of agenda based on user feedback on the minutes of the meeting;
2. The minutes creation support device according to claim 1. an acquisition unit that acquires text data indicating the contents of speeches made in a conference;
a processing unit that acquires a sentence summarizing the speeches at the meeting by inputting, into a large-scale language model, a prompt that differs depending on the type of minutes of the meeting, indicating a degree to which the speeches at the meeting should be summarized, and information based on the text data, for requesting the creation of a sentence summarizing the speeches at the meeting;
an output unit that outputs minutes of the meeting based on the acquired sentences;
A minutes creation support device comprising: On the computer,
A process of acquiring text data indicating the content of speeches made at a meeting;
a process of acquiring a sentence summarizing the speech of the meeting by inputting a prompt including a portion that differs depending on the type of agenda and information based on the text data into a large-scale language model, the prompt being used to request the creation of a sentence summarizing the speech of the meeting;
a process of outputting minutes of the meeting based on the acquired text;
A program to execute.