KR20250063138A - Electronic device for generating training data to be used for training large-scale language model and method for operation thereof - Google Patents

Abstract

According to various embodiments, an electronic device for generating training data to be used for training a large-scale language model includes a display, and a processor, wherein the processor is configured to generate, using an application for generating the training data, at least one of a storage node, a switch node, a branch node, and an output node, generate a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and generate the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message. Other embodiments are also possible.

Description

{ELECTRONIC DEVICE FOR GENERATING TRAINING DATA TO BE USED FOR TRAINING LARGE-SCALE LANGUAGE MODEL AND METHOD FOR OPERATION THEREOF}

Various embodiments of the present disclosure relate to an electronic device and a method of operating the same for generating training data to be used in training a large-scale language model.

Recently, with the technological advancement in the field of artificial intelligence, especially in the field of natural language understanding, the development and utilization of conversational AI agent systems based on large language models (LLMs) that enable users to operate machines and obtain desired services from machines in a more human-friendly manner, such as through conversations using natural language in the form of voice and/or text, rather than through machine operation based on traditional machine-centered command input/output methods, are gradually increasing. Accordingly, in various fields including (but not limited to) online consultation centers and travel service provision platforms, users can now request desired services from conversational AI agent systems and obtain desired results from them through natural language conversations in the form of voice and/or text.

The development of large-scale language models largely relies on the collection and processing of large amounts of text data. This data is usually collected from publicly accessible sources, and the content can be diverse, such as novels, news articles, academic papers, and website content. This data can be fed into a language model, allowing the machine to learn the structure and meaning of the language, and then use this to generate or understand new text.

Existing data collection methods reflect a wide range of language usage, but they can show limitations in domains that require in-depth understanding or specialized knowledge of a specific field. For example, in order to develop language models required in specific specialized fields such as medicine, law, and advanced science and technology, data that accurately reflects the terminology, style, and concepts specific to that field are required. Such data are not easily obtained from general sources, which makes it difficult to develop language models optimized for specific fields.

Korean Patent Publication No. 10-2647511 (2024.03.11) U.S. Patent Publication No. 11,704,506 (2023.07.18)

An electronic device according to the present disclosure can provide a method for generating virtual text data specialized for various domains while being suitable for training a large-scale language model by using rules for generating training data based on probability and state, and automatically expanding the training data by utilizing the large-scale language model.

According to various embodiments, an electronic device for generating training data to be used for training a large-scale language model includes a display, and a processor, wherein the processor is configured to generate, using an application for generating the training data, at least one of a storage node, a switch node, a branch node, and an output node, generate a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and generate the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

According to various embodiments, a method of operating an electronic device for generating training data to be used for training a large-scale language model includes an operation of generating at least one node among a storage node, a switch node, a branch node, and an output node by using an application for generating the training data, an operation of generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and an operation of generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, the one or more programs including instructions for: using an application for generating training data to be used for training a large-scale language model, generating at least one of a storage node, a switch node, a branch node, and an output node; generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node; and generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

In order to train a large-scale language model, a large number of user messages and a large number of agent messages configured in pairs are required. According to the present disclosure, by automatically generating a large amount of training data configured with virtual user messages and virtual agent messages without requiring the user to directly input user messages or agent messages, the training efficiency of a large-scale language model can be improved.

FIG. 1 illustrates a block diagram of a user device and a server according to various embodiments of the present disclosure.
FIG. 2 is a flowchart illustrating operations of an electronic device to generate training data to be used for training a large-scale language model according to various embodiments.
FIG. 3 is a diagram showing a GUI for each type of node used to generate training data according to various embodiments.
FIG. 4 is a diagram showing a node flow object that implements node flow in GUI form according to various embodiments.
FIG. 5 is a diagram illustrating a training data set according to various embodiments.
FIG. 6a is a diagram illustrating a node flow object for generating training data for an LLM for translation according to various embodiments.
FIG. 6b is a diagram illustrating an example of a training data entry of a translation LLM generated by the node flow object of FIG. 6a according to various embodiments.
FIG. 7a is a diagram illustrating a node flow object for generating training data of a summary LLM according to various embodiments.
FIG. 7b is a diagram illustrating an example of a training data entry of a summary LLM generated by the node flow object of FIG. 7a according to various embodiments.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. It should be understood that the embodiments and the terms used herein are not intended to limit the technology described in this document to a specific embodiment, but rather to include various modifications, equivalents, and/or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expression may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B" or "at least one of A and/or B" may include all possible combinations of the items listed together. Expressions such as "first," "second," "first," or "second," may modify the corresponding components, regardless of order or importance, and are only used to distinguish one component from another, but do not limit the corresponding components. When it is said that a certain (e.g., a first) component is "(functionally or communicatively) connected" or "connected" to another (e.g., a second) component, said certain component may be directly connected to said other component, or may be connected through another component (e.g., a third component).

In this document, "configured to" can be used interchangeably with, for example, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to," either hardware-wise or software-wise. In some contexts, the phrase "a device configured to" can mean that the device is "capable of" doing something together with other devices or components. For example, the phrase "a processor configured to perform A, B, and C" can mean a dedicated processor (e.g., an embedded processor) for performing the operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform the operations by executing one or more software programs stored in a memory device.

An electronic device according to various embodiments of the present document may include, for example, at least one of a smartphone, a tablet PC, a desktop PC, a laptop PC, a netbook computer, a workstation, and a server.

Referring to FIG. 1, a user device (100) and a server (101) in various embodiments are described. The user device (100) may include a communication module (110), a processor (120), a memory (130), and a display (140). In some embodiments, the user device (100) may omit at least one of the components or additionally include other components.

The communication module (110) can establish communication between, for example, the user device (100) and an external device (e.g., the first external electronic device (102), the second external electronic device (104), or the server (101)). For example, the communication module (110) can be connected to a network (180) via wireless communication or wired communication to communicate with the external device (e.g., the second external electronic device (104) or the server (101)).

The wireless communication may include, for example, cellular communication using at least one of LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). According to one embodiment, the wireless communication may include, for example, at least one of WiFi (wireless fidelity), Bluetooth, Bluetooth low energy (BLE), Zigbee, near field communication (NFC), Magnetic Secure Transmission, radio frequency (RF), or body area network (BAN). According to one embodiment, the wireless communication may include GNSS. The GNSS may be, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter "Beidou"), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS". The wired communication may include at least one of, for example, a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard232 (RS-232), a power line communication, or a plain old telephone service (POTS). The network (180) may include at least one of a telecommunications network, for example, a computer network (e.g., a LAN or WAN), the Internet, or a telephone network.

The processor (120) may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor (120) may, for example, perform operations or data processing related to control and/or communication of at least one other component of the user device (100).

The memory (130) may include volatile and/or nonvolatile memory. The memory (130) may store, for example, commands or data related to at least one other component of the user device (100).

The display (140) may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display. The display (140) may, for example, display various contents (e.g., text, images, videos, icons, and/or symbols) to a user. The display (160) may include a touch screen and may receive touch, gesture, proximity, or hovering inputs, for example, using an electronic pen or a part of the user's body.

Each of the first and second external electronic devices (102, 104) may be the same or a different type of device as the user device (100). According to various embodiments, all or part of the operations executed in the user device (100) may be executed in another one or more electronic devices (e.g., the electronic devices (102, 104) or the server (101). According to one embodiment, when the user device (100) is to perform a certain function or service automatically or upon request, the user device (100) may request at least some functions related thereto from another device (e.g., the electronic device (102, 104) or the server (101)) instead of executing the function or service by itself or in addition. The other electronic device (e.g., the electronic device (102, 104) or the server (101)) may execute the requested function or additional function and transmit the result to the user device (100). The user device (100) may process the received result as is or additionally to provide the requested function or service. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

The server (101) may include a communication module (111), a processor (121), and a memory (131). In some embodiments, the server (101) may omit at least one of the components or additionally include other components. The communication module (111), the processor (121), and the memory (131) may perform the same functions as the communication module (110), the processor (120), and the memory (130) in the user device (100), respectively.

FIG. 2 is a flowchart illustrating an operation of an electronic device (e.g., a user device (100) of FIG. 1) generating training data to be used for training a large-scale language model according to various embodiments.

FIG. 3 is a diagram showing a GUI for each type of node used to generate training data according to various embodiments.

FIG. 4 is a diagram showing a node flow object that implements node flow in GUI form according to various embodiments.

FIG. 5 is a diagram illustrating a training data set according to various embodiments.

According to various embodiments, the server (101) may operate an application composed of a plurality of execution screens or a website composed of a plurality of web pages, communicate with an electronic device (e.g., an electronic device (100, 102, 104) of FIG. 1) (e.g., a PC, a laptop, a smartphone, etc.) through a network (162, 164), process a request received from a user device (100) regarding an application or a web page, and transmit requested information to the user device (100). The server (101) may store/manage a large-scale language model, and transmit source code for enabling each execution screen of a dedicated application or website for transmitting and receiving messages with the user device (100) through an intelligent agent based on the large-scale language model to the user device (100), and the user device (100) may receive the source code and display an execution screen requested by a user of the user device (100) through the dedicated application or a web browser. According to one embodiment, the server (101) may include the same types of components as the components of the electronic device (100) of FIG. 1. According to one embodiment, the components referred to as user devices (100) in the present disclosure may mean a user account that accesses a platform provided by the server (101) through the user device.

In operation 201, according to various embodiments, an electronic device (100) (e.g., a processor (120) of FIG. 1) may generate at least one of a storage node, a switch node, and an output node using an application for generating training data to be used for training a large language model (LLM).

According to one embodiment, the large-scale language model is a type of generative AI, and may include, for example, a transformer-based neural network model, and may be used in (1) natural language understanding (NLU) fields that understand the meaning of sentences and perform natural language understanding tasks such as keyword extraction, sentiment analysis, and information retrieval, (2) natural language generation (NLG) fields that generate natural language in various forms such as sentences, paragraphs, summaries, and translations, (3) question answering fields that generate answers to given questions, (4) machine translation fields that perform translation tasks between multiple languages, and (5) text summarization fields that concisely summarize long texts to increase the value of information. In addition to the examples described above, the large-scale language model may include various language models that can be easily realized by those skilled in the art.

In one embodiment, the at least one node may be generated in the form of program code or in the form of a graphical user interface through the application. In one embodiment, the application may be an application that provides an integrated development environment capable of programming in various languages (e.g., Visual Studio).

According to one embodiment, the electronic device (100) may display an object corresponding to a node used for generating training data on an application in the form of a GUI through a display (e.g., display (140) of FIG. 1).

According to one embodiment, a storage node used for generating training data may be a component for storing at least one candidate storage variable value for a storage variable. Each of the at least one candidate storage variable value may be selected according to a preset probability value, and each probability value may be set by a user to be the same or different from each other.

According to one embodiment, a switch node used for generating training data may be a component for setting a condition variable representing a branchable condition. According to one embodiment, referring to FIG. 3, a switch node object (311) that implements a switch node in a GUI form may include (i) a node selection object for selecting a type of node (e.g., Switch), (ii) a condition variable value field representing a condition variable (e.g., has_greeted), and (iii) a button for adding a candidate state value of a branch node (e.g., add case).

According to one embodiment, a branch node used for generating training data may be a component for setting a candidate state value that can be branched according to a current state of a condition variable of a switch node. According to one embodiment, referring to FIG. 3, a branch node object (312) that implements a branch node in the form of a GUI may include (i) a candidate state value (e.g., true), (ii) an object for modifying/removing the candidate state value, and (iii) a field for adding another candidate state value, and the branch node object (312) may be implemented as a set (310) with a switch node object (311). The electronic device (100) may transition to a branch node having a value that matches the current condition variable of the switch node. For example, if a switch node is set to a condition variable (e.g., apple_count), a first branch node connected to the switch node is set to a first candidate state value (e.g., 0, 1, 2), and a second branch node connected to the switch node is set to a second candidate state value (e.g., 3), the electronic device (100) can transition to the first branch node when the value of apple_count is the first candidate state value, transition to the second branch node when it is the second candidate state value, and stop execution and generate an error when it is any other value.

According to one embodiment, the output node used for generating training data may be a component for outputting at least one candidate output variable value for an output variable. Each of the at least one candidate output variable value may be selected according to a preset probability value, and each probability value may be set to be the same or different from each other by a user. According to one embodiment, the output node may be set to output a type of the output node, an output variable (name), and content (i.e., a value selected from among the candidate output variable values). According to one embodiment, with reference to FIG. 3, an output node object (320, 330) that implements an output node in a GUI form may include (i) an object that can select a type of the output node (e.g., assistant), (ii) an output variable value field representing an output variable (e.g., GreetingAgain, Greeting), (iii) a candidate output variable value (e.g., "Hi again! I'm {name}."/"Hello again, this is {name}!", "Hi!"/"Hello!"), (iv) an object for modifying/removing the candidate output variable value, and (v) a field for adding another candidate output variable value. According to one embodiment, the type of the output node may be one of prompt data corresponding to a virtual user message (e.g., user) or completion data corresponding to a virtual agent message (e.g., assistant). According to one embodiment, the type of the output node may be set to various types by the user of the electronic device (100) in addition to the prompt data or the completion data.

According to various embodiments, the training data generated according to the present disclosure may include data to be used for training a conversational LLM, data to be used for training a translation LLM, and data to be used for training a summarizing LLM, and may include data to be used for training LLMs for various purposes without being limited to the examples described above. According to one embodiment, the training data may have a pair of properties, such as prompt data and completion data, for each data entry, or may have a single property (e.g., text) or three or more properties (e.g., question/correct answer/wrong answer, etc.).

If each node for generating training data and the node flow including it are implemented in GUI form, it is possible to provide convenience for easily generating training data by intuitively recognizing the node's transition relationship, transition probability, and each setting value.

According to one embodiment, at least one node used for generating training data may further include a void node that does not perform any operation, a start node, and an end node, and the start node and the end node may be expressed as a void node or as a node independent of the void node, and this may be variously changed according to a user's setting. According to one embodiment, the end node may include an object (preview function) that enables displaying a result of generating training data according to a node flow from the start node to the end node. According to one embodiment, the type of at least one node described above is only one example, and various types of nodes for generating training data may be included in addition to a storage node, a switch node, an output node, and a void node according to a setting of a user of the electronic device (100) or an administrator of the server (101).

According to one embodiment, the electronic device (100) can generate at least one node based on a user input. For example, the electronic device (100) can receive a user input to generate a specific node (e.g., an output node) and generate the specific node based on the user input.

In operation 203, according to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may generate a node flow that sets a transition relationship between at least one node and a variable-related value for each of at least one node. According to one embodiment, the node flow may define an execution order of a series of nodes for generating training data, and specifically, may represent a transition relationship and a transition probability between each node from a start node (e.g., a first node) to an end node (e.g., a last node) and a variable-related value within each node. According to one embodiment, the electronic device (100) may implement and display an object corresponding to the node flow in a GUI form on an application through the display (140). For example, FIG. 4 shows a draft for implementing a node flow in a GUI form as a part of a node flow for generating training data of a conversational LLM.

According to various embodiments, an electronic device (100) (e.g., processor (120) of FIG. 1) may establish a transition relationship between at least one node.

According to one embodiment, the electronic device (100) can generate a node flow capable of generating training data by setting a transition relationship and a probability value for the transition relationship that point from one node (e.g., a starting node) to another node (e.g., an arrival node). The starting node and the arrival node merely mean relative concepts between two nodes and do not mean fixed concepts, and each node except for the starting node and the ending node can be an arrival node and also a new starting node. According to one embodiment, the types of the starting node and the arrival node can be the same or different.

According to one embodiment, referring to FIG. 4, the electronic device (100) may set a transition relationship (401) from a storage node (e.g., Set: name) to a storage node (e.g., Set: has_greeted) and a probability value (e.g., 1) for the transition relationship (401).

According to one embodiment, referring to FIG. 4, the electronic device (100) may set a transition relationship (402) from a storage node (e.g., Set: has_greeted) to a switch node (e.g., Switch: has_greeted) and a probability value (e.g., 1) for the transition relationship (402).

According to one embodiment, referring to FIG. 4, the electronic device (100) may set a transition relationship (403) from a branch node (e.g., True) to a storage node (e.g., Set: has_greeted) and a probability value (e.g., 0.05) for the transition relationship (403).

According to one embodiment, referring to FIG. 4, the electronic device (100) may set a transition relationship (404) from a storage node (e.g., Set: has_greeted) to an output node (e.g., User: Greeting) and a probability value (e.g., 1) for the transition relationship (404).

According to one embodiment, referring to FIG. 4, the electronic device (100) may set a transition relationship (405) from an output node (e.g., User: Greeting) to a switch node (e.g., Switch: has_greeted) and a probability value (e.g., 0.5) for the transition relationship (405).

According to one embodiment, referring to FIG. 4, the electronic device (100) may set a transition relationship (406) from a branch node (e.g., True) to an output node (e.g., Assistant: GreetingAgain) and a probability value (e.g., 0.00001) for the transition relationship (406).

According to one embodiment, referring to FIG. 4, the electronic device (100) may set a transition relationship (406) from a branch node (e.g., False) to a void node (e.g., Void: Final) and a probability value (e.g., 0.00001) for the transition relationship (406).

In one embodiment, a transition relationship from a switch node to a branch node may not have a probability value.

According to one embodiment, the node flow may include at least one output node for generating prompt data corresponding to a virtual user message and at least one output node for generating completion data corresponding to a virtual agent message. For example, referring to FIG. 4, the node flow may include a first output node (430) for generating prompt data corresponding to the virtual user message, a second output node (440) for generating completion data corresponding to the virtual first agent message, and a third output node (450) for generating completion data corresponding to the virtual second agent message. Meanwhile, the user message may mean a message obtained from a user, and the agent message may mean a message that is input into a large-scale language model and processed in the form of a response to the user message, and may mean a message output by an intelligent agent (or, intelligent assistant) linked with the large-scale language model. An electronic device (100) according to the present disclosure can automatically generate a large number of virtual user messages and virtual agent messages using node flow without directly receiving training user messages and training agent messages from a user.

According to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may set variable-related values for at least one node each.

According to one embodiment, the electronic device (100) may obtain, from a user, variable-related values for a storage node, including storage variables and candidate storage variable values. For example, referring to FIG. 4, the electronic device (100) may obtain, from a user, storage variables (e.g., name) and candidate storage variable values (e.g., John, Karl, Alice, Alex) for a storage node (410). According to one embodiment, the electronic device (100) may set a probability value for selection of the candidate storage variable value for each candidate storage variable value according to a user input.

According to one embodiment, the electronic device (100) may obtain a condition variable as a variable-related value for a switch node from a user. For example, referring to FIG. 4, the electronic device (100) may obtain a condition variable (e.g., has_greeted) for a switch node (420) from a user.

According to one embodiment, the electronic device (100) may obtain a candidate state value of a condition variable for a switch node (420) as a variable-related value for a branch node from a user. For example, referring to FIG. 4, the electronic device (100) may obtain a candidate state value (e.g., True) of a condition variable (e.g., has_greeted) for a branch node (421) from a user.

According to one embodiment, the electronic device (100) may obtain, from the user, variable-related values for an output node, such as an output variable and a candidate output variable value. For example, referring to FIG. 4, the electronic device (100) may obtain, from the user, an output variable (e.g., User: Greeting) and a candidate output variable value (e.g., Hi, hi, Hello, hello) for an output node (430). According to one embodiment, the electronic device (100) may set a probability value for selecting a candidate output variable value for each candidate output variable value according to a user input.

According to one embodiment, the electronic device (100) can freely perform, regardless of the order of the operations, the setting of transition relationships and variable-related values. For example, the electronic device (100) can set variable-related values for each node after setting transition relationships between nodes, or can set variable-related values for each node after setting transition relationships between nodes.

In operation 205, according to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may generate training data according to a result of executing the node flow in response to a request for generation of training data.

According to one embodiment, after the setting of the transition relationship between each node in the node flow and the setting of the variable-related values for each node are completed, the electronic device (100) may obtain a request for generating training data (i.e., a request for executing the node flow) from a user, and generate training data according to the result of executing the node flow in response to the request. For example, the electronic device (100) may identify the result value of executing the moved node while starting from a start node and proceeding (moving) to the next node according to the transition probability set for each transition relationship, and generate training data according to the result value of executing each node after executing through the last output node to the end node.

According to one embodiment, the electronic device (100) may obtain the number of generated training data (entries) from the user as a request for generating training data. For example, referring to FIG. 5, if the electronic device (100) receives a request to generate 10 training data from the user, the electronic device (100) may generate 10 training data, in which case the first training data represents [{"type":"user", "name":"Greeting", "content":"hello"}, {"type":"assistant", "name":"Greeting", "content":"Hi!"}]. As another example, if the electronic device (100) receives a request to generate 100 training data from the user, the electronic device (100) may generate 100 training data.

According to one embodiment, each training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message. According to one embodiment, the electronic device (100) may output a type, an output variable (name), and content (i.e., a value selected from among candidate output variable values) of an output node corresponding to each of the prompt data and the completion data of the training data according to execution of the node flow. For example, referring to FIG. 5, in the first training data, the prompt data may be {"type":"user", "name":"Greeting", "content":"hello"}, and the completion data may be {"type":"assistant", "name":"Greeting", "content":"Hi!"}.

According to one embodiment, if a candidate output variable value includes a specific storage variable, the electronic device (100) may generate training data with the candidate output variable value including the candidate storage variable value selected for the specific storage variable according to the execution of the node flow. For example, referring to FIG. 3, if a candidate output variable value of an output node object (320) (e.g., "Hi again! I'm {name}.) includes a specific storage variable (e.g., name), the electronic device (100) may generate training data with the candidate output variable value including the candidate storage variable value (e.g., Alex) selected for the specific storage variable according to the execution of the node flow.

According to one embodiment, the electronic device (100) can generate training data in a JSON (JavaScript Object Notation) structured format. The JSON format is only one example and is not limited thereto, and training data can be generated in various formats that can be easily realized by those skilled in the art. For example, the training data can be converted and stored in a file format such as XML (eXtensible Markup Language), CSV (Comma-Separated Values), TXT, etc.

FIG. 6a is a diagram illustrating a node flow object for generating training data for an LLM for translation according to various embodiments.

FIG. 6b is a diagram illustrating an example of a training data entry of a translation LLM generated by the node flow object of FIG. 6a according to various embodiments.

According to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may receive an input from a user to generate at least one node. According to one embodiment, the electronic device (100) may generate an output node object for generating a system message according to a user input, an output node object for generating a virtual user message as prompt data, or an output node object for generating at least one of a virtual agent message as completion data, in order to generate training data of an LLM for translation. For example, referring to FIG. 6A, the electronic device (100) may generate a start node object (601), a first output node object (611) for generating a system message, 2-1st to 2-3rd output node objects (621, 622, 623) for generating a virtual user message, 3-1st to 3-3rd output node objects (631, 632, 633) for generating a virtual agent message, and an end node object (602) according to a user input.

According to one embodiment, a virtual user message as training data of an LLM for translation may mean an original message to be translated, and a virtual agent message may mean a translated message that is a translation of the original message.

According to one embodiment, each of the output node objects for generating a virtual user message and/or a virtual agent message may be linked to an LLM trained to output similar expressions, and the electronic device (100) may expand candidate output variable values by a predetermined number using the LLM according to a user request. For example, if only two data (e.g., “Hello”, “Hello”) are input as candidate output variable values in the output node object (621) for generating a virtual user message, when the user selects an expand button (Expand), the electronic device (100) may input the two data into the LLM to expand similar data (e.g., “Hello~”, “Hello”, “Hello”, etc.) by a predetermined number (e.g., 11) and input them into the output node object (621). Here, the predetermined number may be freely changed and set by the user.

According to one embodiment, the electronic device (100) may receive an input from a user to set a transition relationship and a transition probability between at least one node. For example, referring to FIG. 6A, the electronic device (100) may set a transition relationship and a transition probability (e.g., each 33%) from a first output node object (611) to a second-first to second-third output node objects (621, 622, 623) according to a user input. For another example, referring to FIG. 6A, the electronic device (100) may set a transition relationship and a transition probability (e.g., each 100%) from a second-first to second-third output node objects (621, 622, 623) to a third-first to third-third output node objects (631, 632, 633) according to a user input. As another example, the electronic device (100) can set a transition relationship and a transition probability (e.g., 100% each) from the 3-1 to 3-3 output node objects (631, 632, 633) to the end node object (602) based on user input.

According to one embodiment, the training data of the LLM for translation may be applied with the contents of the training data of the LLM for conversation described in FIGS. 2 to 5, and the training data of the LLM for conversation may also be applied with the contents of the training data of the LLM for translation described above.

According to one embodiment, referring to FIG. 6b, an output node object for generating a virtual user message of training data of an LLM for translation may be set to output one prompt data (e.g., an original message) and one completion data (e.g., a translated message) upon execution of a node flow object, so that the electronic device (100) may generate a training data entry such as FIG. 6b as a result of executing the node flow of FIG. 6a.

FIG. 7a is a diagram illustrating a node flow object for generating training data of a summary LLM according to various embodiments.

FIG. 7b is a diagram illustrating an example of a training data entry of a summary LLM generated by the node flow object of FIG. 6a according to various embodiments.

According to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may receive an input from a user to generate at least one node. According to one embodiment, the electronic device (100) may generate an output node object for generating a system message according to a user input, an output node object for generating a virtual user message as prompt data, or an output node object for generating at least one of a virtual agent message as completion data, in order to generate training data of a summary LLM. For example, referring to FIG. 7A, the electronic device (100) may generate a start node object (701), a first output node object (711) for generating a system message, second-1 to second-2 output node objects (721, 722) for generating a virtual user message, third-1 to third-2 output node objects (731, 732) for generating a virtual agent message, and an end node object (702) according to a user input.

According to one embodiment, a virtual user message as training data of a summary LLM may mean an original message that is a target of summary, and a virtual agent message may mean a summary message that summarizes the original message.

According to one embodiment, each of the output node objects for generating the virtual user message and/or the virtual agent message may be linked to an LLM trained to output a new message according to a predetermined criterion (e.g., an action of outputting the same content in a different manner), and the electronic device (100) may expand the candidate output variable values by a predetermined number using the LLM according to a user request. The above-described predetermined criterion (e.g., an action of outputting the same content in a different manner) may include various actions such as maintaining the content of a sentence but changing the sentence structure/arrangement, maintaining the core expression of the sentence but changing the particles of the sentence, or adding/removing an exclamation mark. For example, if only two data (721a, 721b) are input as candidate output variable values in an output node object (721) for generating a virtual user message, when an expand button (Expand) is selected by the user, the electronic device (100) can input the two data into the LLM to expand new data (e.g., new review messages) that are identical to the contents of the two data (721a, 721b) but written in a different manner by a predetermined number (e.g., three) and input them into the output node object (721). Here, the predetermined number can be freely changed and set by the user.

According to one embodiment, the electronic device (100) may receive an input from a user to set a transition relationship and a transition probability between at least one node. For example, referring to FIG. 7A, the electronic device (100) may set a transition relationship and a transition probability (e.g., each 50%) from a first output node object (711) to a second-first to second-second output node objects (721, 722) according to a user input. For another example, referring to FIG. 7A, the electronic device (100) may set a transition relationship and a transition probability (e.g., each 100%) from a second-first to second-second output node objects (721, 722) to a third-first to third-second output node objects (731, 732) according to a user input. As another example, the electronic device (100) can set a transition relationship and a transition probability (e.g., 100% each) from the 3-1 to 3-2 output node objects (731, 732) to the end node object (702) based on user input.

According to one embodiment, the training data of the summary LLM can be applied with the contents of the training data of the conversational LLM described in FIGS. 2 to 5, and the training data of the conversational LLM can also be applied with the contents of the training data of the summary LLM described above.

According to one embodiment, referring to FIG. 7b, an output node object for generating a virtual user message of training data of a summary LLM may be set to output one prompt data (e.g., an original message) and one completion data (e.g., a summary message) upon execution of a node flow object, and accordingly, the electronic device (100) may generate a training data entry such as FIG. 7b as a result of executing the node flow of FIG. 7a.

According to various embodiments, an electronic device for generating training data to be used for training a large-scale language model includes a display, and a processor, wherein the processor is configured to generate, using an application for generating the training data, at least one of a storage node, a switch node, a branch node, and an output node, generate a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and generate the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

According to various embodiments, the at least one node and the node flow are implemented in the form of a graphical user interface, and the node flow may include at least one output node for generating the prompt data and at least one output node for generating the completion data.

According to various embodiments, the processor may be configured to obtain, as variable-related values for the storage node, a storage variable and a candidate storage variable value, as variable-related values for the switch node, a condition variable, as variable-related values for the branch node, a candidate state value of the condition variable, and as variable-related values for the output node, an output variable and a candidate output variable value.

According to various embodiments, the processor may be configured to generate the training data with candidate output variable values that include a candidate storage variable value selected for the specific storage variable, upon execution of the node flow, if the candidate output variable value includes a specific storage variable.

According to various embodiments, the processor may be configured to output a type, an output variable, and content of an output node corresponding to each of the prompt data and the completion data, according to execution of the node flow.

According to various embodiments, the processor is configured to obtain a number of training data to be generated from a user as a request for generation of the training data, wherein the training data can be generated in a JSON structured format.

According to various embodiments, a method of operating an electronic device for generating training data to be used for training a large-scale language model includes an operation of generating at least one node among a storage node, a switch node, a branch node, and an output node by using an application for generating the training data, an operation of generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and an operation of generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, the one or more programs including instructions for: using an application for generating training data to be used for training a large-scale language model, generating at least one of a storage node, a switch node, a branch node, and an output node; generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node; and generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

The term "module" or "~unit" used in this document includes a unit composed of hardware, software or firmware, and can be used interchangeably with terms such as logic, logic block, component, or circuit, for example. The "module" or "~unit" can be an integrally configured component or a minimum unit or a part thereof that performs one or more functions. The "module" or "~unit" can be implemented mechanically or electronically, and can include, for example, an ASIC (application-specific integrated circuit) chip, FPGAs (field-programmable gate arrays), or a programmable logic device, known or to be developed in the future, that performs certain operations, and can be executed by the processor (120). At least a part of the device (e.g., modules or functions thereof) or the method (e.g., operations) according to various embodiments can be implemented as instructions stored in a computer-readable storage medium (e.g., memory (130)) in the form of a program module. When the above command is executed by a processor (e.g., processor (120)), the processor can perform a function corresponding to the command. The computer-readable recording medium may include a hard disk, a floppy disk, a magnetic medium (e.g., a magnetic tape), an optical recording medium (e.g., a CD-ROM, a DVD, a magneto-optical medium (e.g., a floptical disk), a built-in memory, etc. The command may include a code generated by a compiler or a code executable by an interpreter. A module or a program module according to various embodiments may include at least one or more of the above-described components, some of them may be omitted, or other components may be further included. Operations performed by a module, a program module, or other components according to various embodiments may be executed sequentially, in parallel, repeatedly, or heuristically, or at least some of the operations may be executed in a different order, omitted, or other operations may be added.

And the embodiments disclosed in this document are presented for the purpose of explanation and understanding of the disclosed technical contents, and do not limit the scope of the present disclosure. Therefore, the scope of the present disclosure should be interpreted to include all modifications or various other embodiments based on the technical idea of the present disclosure.

Claims (7)

An electronic device for generating training data to be used for training a large-scale language model,
Display, and
Contains a processor,
The above processor,
Using an application for generating the above training data, at least one node among a storage node, a switch node, a branch node, and an output node is generated,
Generating a node flow that sets a transition relationship between at least one of the nodes and a variable-related value for each of the at least one node, and
In response to the request for generating the above training data, the training data is set to be generated according to the result of executing the above node flow,
The above training data includes prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.
Electronic devices.
In the first paragraph,
The above at least one node and the node flow are implemented in the form of a graphical user interface,
The above node flow comprises at least one output node for generating the prompt data and at least one output node for generating the completion data.
Electronic devices.
In the first paragraph,
The above processor,
As variable-related values for the above storage node, storage variable and candidate storage variable values are obtained,
As a variable related value for the above switch node, a condition variable is obtained,
As a variable related value for the above branch node, obtain a candidate state value of the condition variable, and
As a variable-related value for the above output node, set to obtain output variable and candidate output variable values,
Electronic devices.
In the third paragraph,
The above processor,
If the candidate output variable value includes a specific storage variable, the training data is set to be generated as a candidate output variable value that includes the candidate storage variable value selected for the specific storage variable according to the execution of the node flow.
In the first paragraph,
The above processor,
According to the execution of the above node flow, the type, output variable (name), and content of the output node corresponding to each of the above prompt data and the above completion data are set to be output.
Electronic devices.
In the first paragraph,
The above processor,
As a request for generating the above training data, it is set to obtain the number of training data to be generated from the user,
The above training data is generated in JSON structured format.
Electronic devices.
A method of operating an electronic device for generating training data to be used in training a large-scale language model,
An operation of generating at least one node among a storage node, a switch node, a branch node, and an output node using an application for generating the above training data;
An operation of generating a node flow that sets a transition relationship between at least one of the nodes and a variable-related value for each of the at least one node, and
In response to the request for generating the training data, the operation of generating the training data according to the result of executing the node flow is included.
The above training data includes prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.
How an electronic device operates.

Images