KR102837271B1 - Apparatus and method for generating academic guidance language model, and apparatus for recommending academic guidance information - Google Patents

Abstract

A device and method for generating a career guidance and advancement language model and a device for recommending career guidance and advancement information are disclosed. The device for generating a career guidance and advancement language model according to one embodiment includes: a data collection unit for collecting literature data describing occupations or departments; a graph generation unit for generating a graph reflecting similarities between occupations or departments based on the collected literature data; and a model generation unit for generating a career guidance and advancement language model by pre-training a language model based on the collected literature data and the generated graph.

Description

Apparatus and method for generating academic guidance language model, and apparatus for recommending academic guidance information

It is related to the creation of a career guidance language model and the technology for providing career guidance information using the same.

In general, students worry a lot about their career paths. For example, they worry about what kind of job suits them, what major they should choose, and what kind of high school electives they should choose to get the job that suits them. In most cases, they talk vaguely with their school teachers, parents, and friends, but there are problems that they cannot find a clear answer to. Therefore, most students tend to make arbitrary decisions based on their grades, school preferences, and aptitude.

Korean Patent Publication No. 10-2013-0058568

The purpose is to provide a device and method for generating a career guidance and advancement language model and a device for recommending career guidance and advancement information.

A device for generating a career guidance and advancement language model according to an aspect of the present invention may include: a data collection unit for collecting literature data describing occupations or departments; a graph generation unit for generating a graph reflecting similarities between occupations or departments based on the collected literature data; and a model generation unit for generating a career guidance and advancement language model by pre-training a language model based on the collected literature data and the generated graph.

The above graph generation unit can determine the embedding vector of the collected literature data, determine the similarity between occupations or departments based on the determined embedding vector, express each occupation or each department as a node, and generate the graph by connecting nodes with a similarity greater than a predetermined threshold value as edges.

The above model generation unit can extract occupations or departments whose similarity with each occupation or each department is greater than a predetermined threshold based on the above graph, and pre-train the language model using the literature data describing each occupation or each department and the literature data describing the extracted occupations or departments.

The above model generation unit can randomly mask some tokens from the literature data describing each occupation or each department, and pre-train the language model to match some of the masked tokens based on the masked literature data and the extracted literature data describing the occupation or department.

The above model generation unit can pre-train the language model using the objective function of the following mathematical formula.

[Mathematical formula]

( represents literature data describing an occupation or discipline, Is It represents the skimmed literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and (represents literature data combined with

The above model generation unit can pre-train the language model using the objective function of the following mathematical formula.

[Mathematical formula]

( represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and It represents the combined literature data, is a language model Obtained from Represents the average of the embedding vectors of my tokens, Is It represents literature data whose similarity with is greater than a certain threshold value, is a language model Obtained from (represents the average of the embedding vectors of my tokens)

A device for recommending career guidance and advancement information according to another aspect may include a data collection unit that collects literature data related to career guidance and advancement; a database construction unit that determines an embedding vector of the collected literature data using a career guidance and advancement language model and constructs a career guidance and advancement database based on the embedding vector of each literature data; and an information provision unit that receives information on any one of an occupation, a department, a subject, and a book, and searches the career guidance and advancement database based on the information to extract career guidance and advancement-related information.

The above-mentioned literature data related to career paths may include literature data describing occupations, literature data describing departments, literature data describing subjects, and literature data describing books.

The above database construction unit can determine the similarity between an occupation and a department, an occupation and a subject, an occupation and a book, a department and a subject, a department and a book, or a subject and a book based on the embedding vector of each document data, and match occupations and departments, occupations and subjects, occupations and books, departments and subjects, departments and books, or subjects and books having a similarity higher than a predetermined threshold value, thereby constructing the career guidance and advancement database.

The above information provision unit can extract information on at least one of the occupations, departments, subjects, and books matching the entered information from the career guidance database as career guidance information.

A method for generating a career guidance language model performed by a computing device according to another aspect may include: a step of collecting literature data describing occupations or departments; a step of generating a graph reflecting similarities between occupations or departments based on the collected literature data; and a step of generating a career guidance language model by pre-training a language model based on the collected literature data and the generated graph.

The step of generating the above graph may include determining an embedding vector of the collected literature data, determining similarity between occupations or departments based on the determined embedding vector, expressing each occupation or department as a node, and connecting nodes with a similarity greater than a predetermined threshold value as edges to generate the above graph.

The step of generating the above career guidance language model may include a step of extracting an occupation or department having a similarity with each occupation or each department that is greater than a predetermined threshold based on the graph; and a step of pre-training the language model using literature data describing each occupation or each department and literature data describing the extracted occupation or department.

The step of pre-training the above language model may include a step of randomly masking some tokens from literature data describing each occupation or each department; and a step of pre-training the language model to match some of the masked tokens based on the masked literature data and the extracted literature data describing the occupation or department.

The step of pre-training the above language model can pre-train the language model using the objective function of the following mathematical formula.

[Mathematical formula]

( represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and (represents literature data combined with

The step of pre-training the above language model can pre-train the language model using the objective function of the following mathematical formula.

[Mathematical formula]

( represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and It represents the combined literature data, is a language model Obtained from Represents the average of the embedding vectors of my tokens, Is It represents literature data whose similarity with is greater than a certain threshold value, is a language model Obtained from (represents the average of the embedding vectors of my tokens)

By creating a graph reflecting the similarity between occupations or departments based on literature data describing occupations or departments, and pre-training a language model based on the created graph to create a career guidance language model, a language model specialized for solving various career guidance exploration problems can be created.

In addition, career guidance and advancement information tailored to students can be provided using the career guidance and advancement language model.

Figure 1 is a diagram illustrating a career path language model generation device according to an exemplary embodiment.
Figure 2 is a drawing illustrating a career path information recommendation device according to an exemplary embodiment.
FIG. 3 is a diagram illustrating a method for creating a career path language model according to an exemplary embodiment.
FIG. 4 is a diagram illustrating a method for recommending career path information according to an exemplary embodiment.
FIG. 5 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. When adding reference numerals to components in each drawing, it should be noted that the same components are given the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present invention, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Meanwhile, for each step, each step may occur in a different order than stated, unless the context clearly states a specific order. That is, each step may be performed in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

The terms described below are terms defined in consideration of their functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, their definitions should be made based on the contents throughout this specification.

Although the terms first, second, etc. may be used to describe various components, the components should not be limited by the terms. The terms are used only to distinguish one component from another. The singular expression includes the plural expression unless the context clearly indicates otherwise, and it should be understood that the terms "comprises" or "has" are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the division of components in this specification is only a division by the main function of each component. In other words, two or more components may be combined into one component, or one component may be divided into two or more by more detailed functions. In addition to its own main function, each component may additionally perform some or all of the functions of other components, and some of the main functions of each component may be performed exclusively by other components. Each component may be implemented by hardware or software, or by a combination of hardware and software.

Figure 1 is a diagram illustrating a career path language model generation device according to an exemplary embodiment.

The career path and advancement language model generation device (100) of Fig. 1 may be a device for generating a language model specialized for solving various career path and advancement exploration problems.

Referring to FIG. 1, the career path language model generation device (100) may include a data collection unit (110), a graph generation unit (120), and a model generation unit (130).

The data collection unit (110) can collect literature data describing an occupation or department. For example, the literature data is text data and can include job introduction data or department introduction data, etc.

According to an exemplary embodiment, the data collection unit (110) can collect literature data by receiving a plurality of literature data from a user or receiving a plurality of literature data from an external device using wired or wireless communication technology.

The graph generation unit (120) can generate a graph that reflects similarities between occupations or departments based on collected literature data.

Specifically, the graph generation unit (120) can determine the embedding vector corresponding to each document data by expressing each collected document data on the embedding space. For example, the graph generation unit (120) can determine the embedding vector of each document data by using various word embedding or sentence embedding models such as LSA, Word2Vec, FastText, Glove, and mpnet, but is not limited thereto.

The graph generation unit (120) can determine the similarity between occupations or departments based on the embedding vector of each document data. For example, the graph generation unit (120) can determine the similarity between occupations or departments using various similarity determination techniques such as cosine similarity, Euclidean distance, and Jaccard similarity, but is not limited thereto.

According to one embodiment, the graph generation unit (120) can determine similarity between occupations or departments using mathematical expression 1 based on cosine similarity.

Here, and represents the literature data describing the i-th occupation or department and the literature data describing the j-th occupation or department, respectively. represents an embedding model or embedding function, and are each Obtained from and Represents the embedding vector of , can represent the similarity between the ith occupation or department and the jth occupation or department.

The graph generation unit (120) can generate a graph based on similarity between occupations or departments. At this time, the graph includes nodes and edges, with nodes representing each occupation or department, and edges representing similarity relationships between occupations or departments.

According to an exemplary embodiment, the graph generation unit (120) may generate a graph by connecting only nodes (occupations or departments) with a similarity greater than a predetermined threshold value with edges. However, this is not limited to this, and the graph generation unit (120) may connect all nodes (occupations or departments) with edges of different shapes according to their similarity.

For example, if each occupation or each department is represented as a node in the graph, the ith occupation or department can be considered as the ith node in the graph. The similarity between the ith occupation or department and the jth occupation or department in the graph If an edge exists between two nodes only when the number of edges is greater than or equal to a predetermined threshold value (σ), the generated graph can be expressed by mathematical expression 2.

Here, represents a set of edges, represents a set of occupations or disciplines, can represent a graph.

The model generation unit (130) can generate a career guidance language model by pre-training a language model based on the collected literature data and the generated graph. Here, the language model can have the same or similar structure as BERT (Bidirectional Encoder Representations from Transformers) or a derivative model of BERT (e.g., ALBERT (A Lite BERT for Self-supervised Learning of Language Representations), RoBERTa (A Robustly Optimized BERT Pretraining Approach), ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately), etc.). The career guidance language model is a language model specialized in solving various career guidance exploration problems and can be used to recommend and provide career guidance information customized for students.

According to an exemplary embodiment, the model generation unit (130) may extract occupations or departments whose similarity with each occupation or department is greater than a predetermined threshold based on the generated graph, and may pre-train a language model using the literature data describing each occupation or department and the literature data describing the extracted occupation or department. At this time, the model generation unit (130) may use a masked language modeling technique.

For example, the model generation unit (130) may randomly mask some tokens (e.g., 15% of all tokens) from literature data describing each occupation or department, combine the masked literature data with literature data describing an occupation or department similar to the corresponding occupation or department (having a similarity level higher than a predetermined threshold), and pre-train a language model to match some of the masked tokens based on the combined literature data.

The objective function for this can be expressed as mathematical formula 3.

Here, represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data similar to (similarity level exceeding a certain threshold), Is and It represents the combined literature data, represents an indicator function where masked tokens have a value of 1 and unmasked tokens have a value of 0. Is Represents the tth token of , Is Language model based on Obtained from Represents the embedding vector of the tth token, represents the embedding function, Is Obtained from Represents the embedding vector of , can represent the sum over all possible tokens (∀x, x ∈ V) in a vocabulary V, which is the set of all tokens in the world.

According to an exemplary embodiment, the model generation unit (130) can pre-train the language model so that similar occupations or departments have similar embedding vectors in the embedding space.

The consistency loss function for this can be expressed by mathematical expression 4.

Here, is a language model Obtained from It represents the average of the embedding vectors of the tokens, is a language model Obtained from can represent the average of the embedding vectors of the tokens. and It may be literature data with a similarity level higher than a certain threshold.

According to an exemplary embodiment, the model generation unit (130) may pre-train a language model to match masked tokens based on masked literature data describing each occupation or department and literature data describing occupations or departments similar to the corresponding occupation or department (similarity exceeding a predetermined threshold), while ensuring that similar occupations or departments have similar embedding vectors in the embedding space.

The objective function for this can be expressed by mathematical expression 5.

That is, the model generation unit (130) can generate a career guidance language model by pre-training a language model using the objective function of mathematical expression 3 or mathematical expression 5.

FIG. 2 is a diagram illustrating a career path information recommendation device according to an exemplary embodiment.

The career and academic advancement information recommendation device (200) of Fig. 2 may be a device that recommends and provides career and academic advancement information customized for students using the career and academic advancement language model described above.

Referring to FIG. 2, the career path and advancement information recommendation device (200) may include a data collection unit (210), a database construction unit (220), and an information provision unit (230).

The data collection unit (210) can collect various literature data related to career guidance and advancement. Here, the literature data is text data and can include literature data describing occupations, literature data describing departments, literature data describing subjects, and literature data describing books.

According to an exemplary embodiment, the data collection unit (210) can collect literature data by receiving a plurality of literature data from a user or receiving a plurality of literature data from an external device using wired or wireless communication technology.

The database construction unit (220) can determine the embedding vector of each collected literature data using the career path language model (240).

The database construction unit (220) can determine the similarity between occupations and departments, occupations and subjects, occupations and books, departments and subjects, departments and books, and subjects and books based on the embedding vector of each document data.

For example, the database construction unit (220) can use various similarity judgment techniques such as cosine similarity, Euclidean distance, and Jaccard similarity to determine the similarity between occupations and departments, between occupations and subjects, between occupations and books, between departments and subjects, between departments and books, and between subjects and books, but is not limited thereto.

The database construction unit (220) can build a career and advancement database (250) by matching occupations and departments, occupations and subjects, occupations and books, departments and subjects, departments and books, and subjects and books, whose similarity is greater than a predetermined threshold value. At this time, the predetermined threshold value may be the aforementioned σ, but this is only one example and is not limited thereto.

The information provision unit (230) receives information from a user about one of the occupations, departments, subjects, and books that the user is interested in, searches the career and advancement database (250) based on the information, extracts information about at least one of the occupations, departments, subjects, and books that match the input information as career and advancement information, and provides the extracted career and advancement information to the user.

For example, if subjects b1 and b2, department c1, and book d1 have similarities with occupation a1 that are greater than a predetermined threshold, and information about occupation a1 is input from a user, the information provision unit (230) can search a career guidance database to extract information about subjects b1, b2, department c1, and book d1 that have similarities with occupation a1 that are greater than a predetermined threshold, and provide the extracted information to the user. Through this, the career guidance information recommendation device (200) can provide information about subjects b1, b2, department c1, and book d1 related to occupation a1 to a student interested in occupation a1.

For another example, if a job with a similarity to department c2 exceeds a predetermined threshold value is a2, a subject is b3, and a book is d2, and information about department c2 is input from a user, the information provider (230) can search a career and advancement database to extract information about job a2, subject b3, and book d2 that exceed a predetermined threshold value in similarity to department c2, and provide the extracted information to the user. Through this, the career and advancement information recommendation device (200) can provide information about job a2, subject b3, and book d2 related to department c2 to a student interested in department c2.

Meanwhile, according to an exemplary embodiment, the data collection unit (210) can periodically collect new literature data related to career guidance and advancement, and in this case, the database construction unit (220) can periodically update the existing career guidance and advancement database (250) using the new literature data.

FIG. 3 is a diagram illustrating a method for creating a career path language model according to an exemplary embodiment.

The method for generating a career path language model of Fig. 3 can be performed by the career path language model generating device of Fig. 1.

Referring to Figure 3, the career path language model generation device can collect literature data describing occupations or departments (310). For example, the literature data is text data and can include job introduction data or department introduction data, etc.

According to an exemplary embodiment, a career path language model generation device can collect literature data by receiving a plurality of literature data from a user or receiving a plurality of literature data from an external device using wired or wireless communication technology.

The career path language model generation device can generate a graph reflecting similarities between occupations or departments based on collected literature data (320).

Specifically, the career path language model generation device can represent each collected document data in an embedding space, and determine an embedding vector corresponding to each document data. For example, the career path language model generation device can determine an embedding vector of each document data using various word embedding or sentence embedding models such as LSA, Word2Vec, FastText, Glove, and mpnet, but is not limited thereto.

In addition, the career path language model generation device can determine the similarity between occupations or departments based on the embedding vector of each document data, and can generate a graph based on the similarity between occupations or departments. At this time, the graph includes nodes and edges, and the nodes can express each occupation or each department, and the edges can express the similarity relationship between occupations or departments.

According to an exemplary embodiment, the career guidance and advancement language model generation device can generate a graph by connecting only nodes (occupations or departments) with similarities higher than a predetermined threshold value with edges. However, this is not limited to this, and the career guidance and advancement language model generation device can connect all nodes (occupations or departments) with edges of different shapes according to their similarities.

The device for generating a career guidance and advancement language model can generate a career guidance and advancement language model by pre-training a language model based on collected literature data and generated graphs (330). Here, the language model can include BERT (Bidirectional Encoder Representations from Transformers) and derivative models of BERT (e.g., ALBERT (A Lite BERT for Self-supervised Learning of Language Representations), RoBERTa (A Robustly Optimized BERT Pretraining Approach), ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately), etc.). The career guidance and advancement language model is a language model specialized in solving various career guidance and advancement exploration problems and can be used to recommend and provide career guidance and advancement information customized for students.

According to an exemplary embodiment, a career guidance and advancement language model generation device can extract occupations or departments whose similarity with each occupation or department is greater than a predetermined threshold based on the generated graph, and pre-train a language model using literature data describing each occupation or department and literature data describing the extracted occupations or departments.

For example, a language model creation device for career guidance and advancement can randomly mask some tokens (e.g., 15% of all tokens) from literature data describing each occupation or department, combine the masked literature data with literature data describing occupations or departments similar to the relevant occupation or department (having a similarity level higher than a predetermined threshold), and pre-train a language model to match some of the masked tokens based on the combined literature data.

As another example, a language model generation device for career guidance can pre-train a language model so that similar occupations or departments have similar embedding vectors in the embedding space.

As another example, the career guidance and advancement language model generation device can pre-train the language model so that masked tokens are matched based on masked literature data describing each occupation or department and literature data describing occupations or departments similar to the relevant occupation or department (with a similarity level higher than a predetermined threshold), while similar occupations or departments have similar embedding vectors in the embedding space.

That is, the career guidance and advancement language model generation device can generate a career guidance and advancement language model by pre-training a language model using the objective function of mathematical expression 3 or mathematical expression 5.

FIG. 4 is a diagram illustrating a method for recommending career path information according to an exemplary embodiment.

The career and academic advancement information recommendation method of Fig. 4 can be performed by the career and academic advancement information recommendation device of Fig. 2.

Referring to Figure 4, the career and advancement information recommendation device can collect various literature data related to career and advancement (410). Here, the literature data is text data and can include literature data describing occupations, literature data describing departments, literature data describing subjects, and literature data describing books.

The career and advancement information recommendation device can determine the embedding vector of each collected literature data using the career and advancement language model (420).

The career and academic advancement information recommendation device can determine the similarity between occupations and departments, occupations and subjects, occupations and books, departments and subjects, departments and books, and subjects and books based on the embedding vector of each literature data (430).

For example, a career guidance and education information recommendation device can use various similarity judgment techniques such as cosine similarity, Euclidean distance, and Jaccard similarity to determine the similarity between occupations and departments, between occupations and subjects, between occupations and books, between departments and subjects, between departments and books, and between subjects and books, but is not limited thereto.

The career guidance and advancement information recommendation device can build a career guidance and advancement database by matching occupations and departments, occupations and subjects, occupations and books, departments and subjects, departments and books, and subjects and books with similarities exceeding a predetermined threshold value (440). At this time, the predetermined threshold value may be the aforementioned σ, but this is only one example and is not limited thereto.

A career guidance and advancement information recommendation device may receive information from a user about one of the occupations, departments, subjects, and books that the user is interested in, search a career guidance and advancement database based on the information, extract information about at least one of the occupations, departments, subjects, and books that match the input information as career guidance and advancement information, and provide the extracted career guidance and advancement information to the user (450).

FIG. 5 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments. In the illustrated embodiment, each of the components may have different functions and capabilities other than those described below, and may include additional components other than those described below.

The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a career guidance and advancement language model generation device (100) or a career guidance and advancement information recommendation device (200).

A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate in accordance with the exemplary embodiments described above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations in accordance with the exemplary embodiments.

A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and/or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and capable of storing desired information, or a suitable combination thereof.

A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).

The computing device (12) may also include one or more input/output interfaces (22) that provide interfaces for one or more input/output devices (24) and one or more network communication interfaces (26). The input/output interfaces (22) and the network communication interfaces (26) are coupled to the communication bus (18). The input/output devices (24) may be coupled to other components of the computing device (12) via the input/output interfaces (22). Exemplary input/output devices (24) may include input devices such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and/or photographing devices, and/or output devices such as a display device, a printer, speakers, and/or a network card. The exemplary input/output devices (24) may be included within the computing device (12) as a component that constitutes the computing device (12), or may be coupled to the computing device (12) as a separate device distinct from the computing device (12).

The present invention has been described so far with reference to preferred embodiments thereof. Those skilled in the art will understand that the present invention can be implemented in modified forms without departing from the essential characteristics of the present invention. Accordingly, the scope of the present invention should not be limited to the above-described embodiments, but should be interpreted to include various embodiments within a scope equivalent to the contents described in the patent claims.

100: Career Path Language Model Generation Device
110, 210: Data Collection Unit
120: Graph generation section
130: Model Generation Section
200: Career and Academic Advancement Information Recommendation Device
220: Database Construction Department
230: Information Department
240: Career Path Language Model
250: Career Path Database

Claims (16)

A data collection unit that collects literature data describing occupations or disciplines;
A graph generation unit that generates a graph reflecting similarities between occupations or departments based on the collected literature data; and
A model generation unit that generates a career guidance and advancement language model by pre-training a language model based on the collected literature data and the generated graph; including,
The above model generation unit,
Based on the above graph, occupations or departments whose similarity with each occupation or each department is greater than a predetermined threshold are extracted, and the language model is pre-trained using the literature data describing each occupation or each department and the literature data describing the extracted occupation or department.
Randomly masking some tokens from the literature data describing each of the above occupations or departments, combining the masked literature data with the literature data describing the extracted occupations or departments, and pre-training the language model to match some of the masked tokens based on the combined literature data.
A device for generating a language model for career advancement. In the first paragraph,
The above graph generation unit,
The embedding vector of the collected literature data is determined, and the similarity between occupations or departments is determined based on the determined embedding vector, and each occupation or department is expressed as a node, and nodes with a similarity greater than a predetermined threshold are connected as edges to generate the graph.
A device for generating language models for career advancement. delete delete In the first paragraph,
The above model generation unit,
Pre-training the language model using the objective function of the following mathematical formula.
A device for generating a language model for career advancement.
[Mathematical formula]

( represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and (represents literature data combined with In the first paragraph,
The above model generation unit,
Pre-training the language model using the objective function of the following mathematical formula.
A device for generating a language model for career advancement.
[Mathematical formula]

( represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and It represents the combined literature data, is a language model Obtained from Represents the average of the embedding vectors of my tokens, Is It represents literature data whose similarity with is greater than a certain threshold value, is a language model Obtained from (represents the average of the embedding vectors of my tokens) Data collection department that collects literature data related to career paths and advancement;
A database construction unit that determines the embedding vector of the collected literature data using the career path and advancement language model and constructs a career path and advancement database based on the embedding vector of each literature data; and
An information provision unit that inputs information on one of occupation, department, subject, and book, searches the career and advancement database based on the information, and extracts career and advancement-related information; including;
The above career guidance and advancement language model is a career guidance and advancement information recommendation device, which is pre-learned to match some of the masked tokens based on literature data that is generated by randomly masking some tokens from literature data describing each occupation or each department and literature data that describes occupations or departments having a similarity with each occupation or each department above a predetermined threshold value. In Article 7,
The literature data related to the above career path is as follows:
Including literature data describing occupations, literature data describing departments, literature data describing subjects, and literature data describing books.
Career and academic advancement information recommendation device. In Article 7,
The above database construction unit,
Based on the embedding vector of each literature data, the similarity between an occupation and a department, an occupation and a subject, an occupation and a book, a department and a subject, a department and a book, or a subject and a book is determined, and the occupation and department, occupation and subject, occupation and book, department and subject, department and book, or subject and book with a similarity higher than a predetermined threshold are matched to build the career and advancement database.
A device that recommends career and academic information. In Article 9,
The above information provider is,
Extracting information on at least one of the occupations, departments, subjects, and books matching the entered information from the career guidance database as career guidance information.
Career and academic advancement information recommendation device. In a method for generating a career guidance language model performed by a computing device,
A step of collecting literature data describing an occupation or discipline;
A step of generating a graph reflecting similarity between occupations or departments based on the collected literature data; and
A step of generating a career guidance and advancement language model by pre-training a language model based on the collected literature data and the generated graph; including,
The steps for creating the above career path language model are:
A step of extracting occupations or departments whose similarity with each occupation or each department is greater than a predetermined threshold based on the above graph; and
It includes a step of pre-training the language model using literature data describing each occupation or each department and the extracted literature data describing the occupation or department;
The step of pre-training the above language model is:
A step of randomly masking some tokens from the literature data describing each occupation or each discipline;
A step of combining the above masked literature data with the above extracted literature data describing the occupation or department; and
A method for creating a career advancement language model, comprising the step of pre-training the language model to match some of the masked tokens based on the combined literature data. In Article 11,
The steps for generating the above graph are:
The embedding vector of the collected literature data is determined, and the similarity between occupations or departments is determined based on the determined embedding vector, and each occupation or department is expressed as a node, and nodes with a similarity greater than a predetermined threshold are connected as edges to generate the graph.
How to create a career path language model. delete delete In Article 11,
The step of pre-training the above language model is:
Pre-training the language model using the objective function of the following mathematical formula.
How to create a career path language model.
[Mathematical formula]

( represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and (represents literature data combined with In Article 11,
The step of pre-training the above language model is:
Pre-training the language model using the objective function of the following mathematical formula.
How to create a career path language model.
[Mathematical formula]

( represents literature data describing an occupation or discipline, Is Represents masked literature data, Is It represents literature data whose similarity with is greater than a certain threshold value, Is and It represents the combined literature data, is a language model Obtained from Represents the average of the embedding vectors of my tokens, Is It represents literature data whose similarity with is greater than a certain threshold value, is a language model Obtained from (represents the average of the embedding vectors of my tokens)