CN118796191A - Code parsing method, device, system, electronic device and storage medium - Google Patents

Abstract

The invention provides a code analysis method, a device, a system, electronic equipment and a storage medium, and relates to the technical field of code processing. Comprising the following steps: determining a code file to be operated in the project from the file according to the project, dividing codes in the code file into segments taking functions as minimum units, and obtaining a plurality of code segments; determining the code hierarchy relation corresponding to the plurality of code segments according to the calling relation of the functions in the code file; according to the sequence from low to high of the code levels, a plurality of code segments are sequentially input into a first natural language model according to a first preset instruction format to obtain code annotations corresponding to the code segments, the first preset instruction format is used for instructing the first natural language model to generate the code annotations of the code segments, and by inputting the code segments into the first natural language model according to the sequence from low to high of the code levels, the first natural language model can generate more accurate annotations by using calling relations among the code segments, so that quick code interpretation is realized.

Description

Code parsing method, device, system, electronic device and storage medium

Technical Field

The present invention relates to the field of code processing technology, and in particular to a code parsing method, device, system, electronic device and storage medium.

Background Art

There are many open source GitHub projects on the AI platform. The content in the README files of many GitHub projects is neither detailed nor relevant to the code. The project comments are mainly about function input and output, and there is a lack of corresponding comments and explanations for many core statements in the functions.

Currently, for the code that needs to be run in the project written in the README file, people mostly rely on reading the code repeatedly to understand the project implementation logic. This method is inefficient.

Summary of the invention

The present invention provides a code parsing method, device, system, electronic device and storage medium, which are used to solve the defect of relying on manual repeated reading of codes to understand the project implementation logic in the prior art, resulting in low processing efficiency.

The present invention provides a code parsing method, the method comprising:

Determine the code files that need to be run in the project according to the project readme file, divide the codes in the code files into segments with functions as the smallest units, and obtain a plurality of code segments;

Determining the code hierarchical relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file;

In order from low to high code levels, several code segments are input into the first natural language model in turn according to a first preset instruction format to obtain code comments corresponding to each code segment. The first preset instruction format is used to instruct the first natural language model to generate code comments for the code segments. The first natural language model is obtained by training an initial natural language model based on the original code segments that identify the original code comments.

According to a code parsing method provided by the present invention, the method further includes:

Determine the literature cited in the project README file, and obtain the literature file in a preset format corresponding to the literature;

Dividing the content in the document file into a plurality of sentences according to preset delimiters;

sequentially inputting a plurality of the sentences into a second natural language model according to a second preset instruction format to obtain a literature answer corresponding to each of the sentences, wherein the second preset instruction format is used to instruct the second natural language model to generate a literature answer corresponding to the sentence;

Based on the text similarity between the code annotation corresponding to the code segment and the literature answer corresponding to the sentence, an association between the code segment and the sentence is established to assist in code interpretation.

According to a code parsing method provided by the present invention, the method further includes:

Processing the text information in the project README file in blocks to obtain a plurality of text blocks;

Obtain an embedding vector set, wherein the embedding vector set includes an embedding vector of each of the code segments, an embedding vector of each of the sentences, and an embedding vector of each of the text blocks;

When receiving a query statement from a user, obtaining an embedding vector of the query statement;

According to a first similarity between the embedding vector of the query sentence and each embedding vector in the embedding vector set, selecting a target embedding vector with the highest first similarity from the embedding vector set;

According to the target embedding vector, answer information corresponding to the query statement is obtained.

According to a code parsing method provided by the present invention, obtaining answer information corresponding to the query statement according to the target embedding vector includes:

Obtaining a target text represented by the target embedding vector;

The target text is used as context information and is input into a third natural language model together with the query statement to obtain answer information corresponding to the query statement.

According to a code parsing method provided by the present invention, determining the literature cited by the project README file and obtaining a literature file in a preset format corresponding to the literature includes:

Matching the text information of the references in the project README file based on a preset regular expression;

Parsing meta information of the cited document based on the text information;

A text file in a preset format is obtained from an open source document library based on the meta information.

According to a code parsing method provided by the present invention, the first natural language model is trained in the following manner:

Crawl the original code snippets with original code comments;

Inputting the original code segment into an initial natural language model to obtain a predicted annotation output by the initial natural language model;

According to the second similarity between the predicted annotation and the original code annotation, the model parameters of the initial natural language model are updated until the second similarity reaches a preset similarity requirement, thereby obtaining the trained first natural language model.

The present invention also provides a code parsing device, the device comprising:

The first code parsing module is used to determine the code files that need to be run in the project according to the project readme file, and divide the codes in the code files into segments with functions as the smallest unit to obtain a plurality of code segments;

A second code parsing module, used to determine the code hierarchical relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file;

The third code parsing module is used to input the plurality of code segments into the first natural language model in sequence according to a first preset instruction format in order from low to high code levels to obtain code comments corresponding to each code segment, wherein the first preset instruction format is used to instruct the first natural language model to generate code comments for the code segments, and the first natural language model is obtained by training an initial natural language model based on the original code segments that identify the original code comments.

The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the code parsing methods described above is implemented.

The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the code parsing method described in any one of the above is implemented.

The present invention also provides a computer program product, comprising a computer program, which implements any of the above-mentioned code parsing methods when executed by a processor.

The code parsing method, device, system, electronic device and storage medium provided by the present invention first determine the code file that needs to be run in the project according to the project self-description file, divide the code in the code file into fragments with functions as the smallest unit to obtain a plurality of code segments, and then determine the code hierarchy relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file, so as to build a bottom-up logical system through the code hierarchy relationship, and then input the plurality of code segments into the first natural language model in sequence according to the first preset instruction format in the order from low to high code hierarchy. By inputting the code segments into the first natural language model in the order from low to high code hierarchy, the first natural language model can better understand the context and logical flow of the code, so that the first natural language model can use the code hierarchy relationship between the code segments to generate more accurate annotations, thereby realizing rapid code interpretation.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic diagram of a flow chart of a code parsing method provided by the present invention;

FIG2 is a schematic diagram of the structure of a code parsing device provided by the present invention;

FIG. 3 is a schematic diagram of the structure of an electronic device provided by the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

There are currently many open source GitHub projects on the artificial intelligence platform. GitHub projects refer to software projects stored on the GitHub code hosting platform. GitHub is a version control and collaboration platform for developers that allows developers to share, host and collaborate on code development. On GitHub, developers can create their own code repository and host the code in the repository. Each code repository corresponds to a project, and developers can perform operations such as code management, version control, and issue tracking in the repository. Other developers can browse and clone these repositories, or participate in the project, submit code, report problems, or provide suggestions for improvements. GitHub projects usually consist of multiple code files, including source code, configuration files, documents, and other related resources.

Currently, the content written in the README files of many GitHub projects is neither detailed nor relevant to the code. Project comments are mostly focused on function input and output, and there is a lack of corresponding comments and explanations for many core statements within the function.

For the code that needs to be run in the project written in the README file, people often rely on reading the code repeatedly to understand the project implementation logic. This method is inefficient and has a high error rate.

In response to the above problems, an embodiment of the present invention proposes a code parsing method, by inputting several code segments that need to be run in the project into a first natural language model in sequence according to a first preset instruction format in an order from low to high code hierarchy. By inputting the code segments into the first natural language model in an order from low to high code hierarchy, the first natural language model can utilize the code hierarchy relationship between the code segments to generate more accurate annotations, thereby achieving rapid code interpretation.

In addition, some GitHub projects are also associated with some literature, such as papers, but the mathematical inferences in the papers are not associated with the implementation part in the project code. Many GitHub projects consider the simplicity of engineering implementation when implementing them, and the content written in its README file may not be completely consistent with the solution written in the paper.

There are already some tools for document interpretation. These tools generally parse the text content in the document first, and then guide the model to interpret by adding appropriate prompts and the text content of the corresponding document. This part of the interpretation usually focuses on translation, summary generation, and text generation. The correlation between the parsing results and the project code is relatively small, which makes the interpretation more difficult.

Therefore, in response to the above problems, an embodiment of the present invention further proposes to fine-tune the model parameters of the first natural language model based on the first natural language model to obtain a second natural language model, and then input several sentences in the literature cited in the project self-described file into the second natural language model according to a second preset instruction format to obtain the literature answers corresponding to the sentences, and use the text similarity between the code annotation corresponding to the code segment and the literature answer corresponding to the sentence to establish an association between the code segment and the sentence to assist in code interpretation.

In addition, the embodiment of the present invention also proposes a method for answering questions about project codes raised by users, which specifically answers users' questions, realizes customized code analysis, and improves answering efficiency.

FIG. 1 is a flow chart of a code parsing method provided by the present invention. As shown in FIG. 1 , the code parsing method includes:

Step 101, determining the code files that need to be run in the project according to the project readme file, dividing the codes in the code files into segments with functions as the smallest unit, and obtaining a plurality of code segments;

Among them, the project readme file refers to the README file, which usually includes the project name (usually with the project name as the title or beginning to clearly identify the project), project description (including its purpose, functions and features, etc. These descriptions can help readers understand the basic background and goals of the project), installation instructions (including detailed guidelines on how to install and configure the project, such as required software dependencies, environment settings and steps, etc.), usage instructions (usually provide instructions on how to use the project, including steps to start the project, run sample code, or perform specific operations, etc.), file structure (README file may list the file structure of the project, including the functions and roles of each file and folder. This helps readers better understand the organization and code layout of the project), examples and demonstrations (including sample code, command line examples or demonstration links to show the functions and usage of the project. These examples can help users better understand and verify the functions of the project), license information (including the license information of the project, clarifying the license agreement used by the project).

In the present embodiment, the README file is parsed to extract the code file that needs to be run, and appropriate regular expressions can be used to match and extract the code file information. For example, it can be matched according to the file name extension (such as .py) or special tags (such as: code file) or function name, parameter list and function body. The file list matched by the regular expression can be used as a candidate code file, and natural language processing techniques such as text segmentation, keyword extraction, etc. can also be used to identify and extract text information related to the code file. For example, sentences or paragraphs containing keywords (such as "code", "run", "file", etc.) can be found, and file name or path information can be extracted therefrom.

After extracting the candidate code files, further organize and screen the candidate code files to ensure that only the code files that actually need to be run are included. For example, according to the requirements of the project, remove files that do not need to be run, such as document files, configuration files, etc. Furthermore, duplicate files can be removed from the candidate code files to avoid repeatedly running the same code. In addition, code files related to the core functions of the project can be further screened out based on the description or instructions provided in the README file.

It should be noted that a code segment refers to a continuous portion of code in a program. It can be an independent function, method, or a specific logical operation or algorithm implementation. In this embodiment, the code can be divided into segments with functions as the smallest unit according to the logical structure and function of the code.

For example, the code can be divided into functions or methods based on their definitions and calling relationships. A code segment can contain a complete function or method, which encapsulates specific functionality and can be called by other code. In addition, the code can be divided into multiple code segments based on the grammatical rules of the programming language, such as based on grammatical elements such as statements, expressions, loops, or conditional statements. The code can also be divided into multiple code segments by identifying the start and end of a code block, but it should be noted that there must be at least one function segment in each of the divided code segments.

Step 102, determining the code hierarchy relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file;

It should be noted that the calling relationship of functions in a code file refers to the mutual calling relationship between functions involved in the code in the code file, that is, one function calls other functions during execution. In a program, a function can call other functions by using the function name and providing the required parameters.

The calling relationship of functions can form a function call graph, which represents the dependency and calling process between functions. In the function call graph, each function can be regarded as a node, and the calling relationship between functions can be regarded as the edge between nodes. By analyzing the calling relationship of functions, the execution process, dependency and logical structure of the program can be clarified.

In this embodiment, the code hierarchical relationship corresponding to each code segment is determined by the mutual calling relationship between functions, such as the following three code segments:

By analyzing the code function calling function, we can know that print("Hello, World!") is the lowest code segment. Since it does not call other functions, it is at the lowest level. function2() is called by function1(), so function2() is at a higher level than function1(). function1() is called by main(), so function1() is at a higher level than main(). main() is the top-level code segment because it is not called by other functions.

Step 103, in order from low to high code levels, input the plurality of code segments into the first natural language model in turn according to a first preset instruction format to obtain code comments corresponding to each of the code segments, wherein the first preset instruction format is used to instruct the first natural language model to generate code comments for the code segments, and the first natural language model is obtained by training an initial natural language model based on the original code segments that identify the original code comments.

For example, for code snippet 1:

pythonCopy Code

print("Hello, World!")

The code segment 1 may be input into the first natural language model according to the following first preset instruction format:

"instruct":"Please generate code comments for the following code snippet:",

"code":"print('Hello, World!')"

By converting the code to the instruction format, we can explicitly specify the task or question when interacting with the first natural language model and expect the model to generate responses related to it.

In this embodiment, in order to improve the accuracy of the code comments parsed by the first natural language model, the code segments are input into the first natural language model in sequence according to the first preset instruction format in order from low to high code levels.

In addition, when the code segments are input into the first natural language model according to the first preset instruction format, specific tags or comments can be used to represent the code hierarchy relationship between the code segments, or specific function names or prefixes can be used to imply the code hierarchy relationship, or the code hierarchy relationship can be expressed through function parameter passing, etc., to help the natural language model understand the function and calling relationship of each code segment, so as to better parse the code comments corresponding to the code segments.

The code parsing method provided by the embodiment of the present invention first determines the code files that need to be run in the project according to the project self-description file, divides the codes in the code files into fragments with functions as the smallest units to obtain a number of code segments, and then determines the code hierarchy relationship corresponding to the number of code segments according to the calling relationship of the functions in the code file, so as to build a bottom-up logical system through the code hierarchy relationship, and then, in order from low to high code levels, the number of code segments are input into the first natural language model in sequence according to a first preset instruction format for representing the calling relationship between the number of code segments. By inputting the code segments into the first natural language model in order from low to high code levels, the first natural language model can better understand the context and logical flow of the code, so that the first natural language model can use the calling relationship between the code segments to generate more accurate annotations, thereby realizing rapid code interpretation.

Based on any of the foregoing embodiments, the first natural language model is trained in the following manner:

Crawl the original code snippets with original code comments;

Inputting the original code segment into an initial natural language model to obtain a predicted annotation output by the initial natural language model;

According to the second similarity between the predicted annotation and the original code annotation, the model parameters of the initial natural language model are updated until the second similarity reaches a preset similarity requirement, thereby obtaining the trained first natural language model.

In this embodiment, for example, the original code segments containing the original code comments can be crawled from the GitHub website and used as training data. These original code segments are then input using the initial natural language model and predicted comments are generated. Based on the similarity between the predicted comments and the original code comments, the model parameters are updated using the back propagation algorithm until the required comment similarity reaches a preset value. Through this process, we can obtain a trained first natural language model that can be used to generate appropriate comments when a code segment is given.

In this embodiment, the similarity between the predicted comments and the original code comments can be calculated based on the similarity calculation method of word vector or text embedding, and there is no limitation on this.

Based on any of the above embodiments, the method further includes:

Determine the literature cited in the project README file, and obtain the literature file in a preset format corresponding to the literature;

Dividing the content in the document file into a plurality of sentences according to preset delimiters;

sequentially inputting a plurality of the sentences into a second natural language model according to a second preset instruction format to obtain a literature answer corresponding to each of the sentences, wherein the second preset instruction format is used to instruct the second natural language model to generate a literature answer corresponding to the sentence;

Based on the text similarity between the code annotation corresponding to the code segment and the literature answer corresponding to the sentence, an association between the code segment and the sentence is established to assist in code interpretation.

In this embodiment, the content in the project narration file is parsed to determine the literature cited by the content in the project narration file. After the cited literature is determined, the literature file in a preset format is downloaded from the corresponding literature library.

Usually, the project self-reading file will contain a reference list or a section for cited literature. The cited literature can be determined based on the content of this section. You can also search for literature-related information in the project self-reading file, such as keywords, authors, journals or conference names, etc., and the cited literature can be determined based on such information.

The preset delimiter refers to a pre-set symbol that can be used to represent a complete paragraph or a complete sentence, such as a period, a question mark, an exclamation mark, a semicolon, a line break, etc.

In this embodiment, the document file is segmented using the selected preset separator. According to the preset separator, the characters are split into multiple parts at corresponding positions to form a sentence list.

It should be noted that the second natural language model used in this embodiment has the same model network structure as the first natural language model, but different model parameters. The second natural language model is a natural language model obtained by training the first natural language model with the first natural language model as the initial model using sentences that identify literature answers. In this way, the second natural language model is obtained by fine-tuning the parameters of the first natural language model, and the second natural language model is used to generate literature answers for sentences in the literature cited in the project's self-described file. This can maintain the consistency of the knowledge domain of the literature answers and the code comments, and improve readability and comprehensibility.

In this embodiment, the sentence is input into the second natural language model according to a second preset instruction format to instruct the second natural language model to generate a document answer corresponding to the sentence.

In addition, in order to establish the association between the code segment and the sentence, in this embodiment, the text similarity algorithm (such as cosine similarity, etc.) can be used to calculate the similarity between the document answer corresponding to the sentence and the code comment corresponding to each code segment, and a suitable similarity threshold is set according to the actual situation to determine whether the code segment and the sentence are related. If the similarity exceeds the threshold, they are considered to be related; otherwise, they are considered to be unrelated. If they are related, the current sentence and the corresponding code segment are associated, for example, stored in the form of a list.

The code parsing method provided by the embodiment of the present invention is based on the first natural language model, and the model parameters of the first natural language model are fine-tuned to obtain a second natural language model. Then, several sentences in the literature cited in the project self-described file are input into the second natural language model according to a second preset instruction format to obtain the literature answers corresponding to the sentences, and the text similarity between the code annotation corresponding to the code segment and the literature answer corresponding to the sentence is used to establish an association between the code segment and the sentence to assist in code interpretation.

Based on any of the above embodiments, the method further includes:

Processing the text information in the project README file in blocks to obtain a plurality of text blocks;

Obtain an embedding vector set, wherein the embedding vector set includes an embedding vector of each of the code segments, an embedding vector of each of the sentences, and an embedding vector of each of the text blocks;

When receiving a query statement from a user, obtaining an embedding vector of the query statement;

According to a first similarity between the embedding vector of the query sentence and each embedding vector in the embedding vector set, selecting a target embedding vector with the highest first similarity from the embedding vector set;

According to the target embedding vector, answer information corresponding to the query statement is obtained.

A text block usually refers to a continuous text wrapped in a certain format or markup. These text blocks are usually used to carry a specific content or information, such as a project introduction, installation guide, usage examples, contribution guide, etc.

In the present embodiment, when the text information in the project README file is processed in blocks, since there are usually different titles or sections in the project README file, the text information can be divided into different blocks by identifying these titles. For example, the title and subtitle can be used as the name of each block, and all the text content below them can be used as the content of the block. In addition, the project README file usually contains information such as project characteristics, system requirements, installation instructions, etc., which can be divided into blocks according to lists or paragraphs. For example, different blocks can be determined by judging the starting and ending positions of each paragraph or list item. In addition, if the project README file uses tags or other special formats to mark different parts, it can also be divided into blocks according to these tags. For example, in the README file using Markdown format, blank lines or specific symbols (such as #) are used to define titles, code blocks, lists, etc., so that the text is divided into different blocks.

In this embodiment, for the text information in the project README file, each divided text block is taken as a unit and input into the model to obtain the corresponding embedding vector; for the code in the code file, each divided code segment is taken as a unit and input into the model to obtain the corresponding embedding vector; for the literature cited in the project README file, the literature is divided into multiple sentences at the preset separators (such as period, semicolon, exclamation mark, etc.) according to the structure of the sentences in the literature. Then each sentence is taken as a unit and input into the model to obtain the corresponding embedding vector.

When a user's query statement is received, the same pre-trained model and corresponding text processing tools are used to take the user's query statement as input and convert it into an embedding vector of the query statement. Then, the first similarity between the embedding vector of the query statement and the embedding vectors of each code segment, sentence, and text block in the embedding vector set is calculated to find the target embedding vector with the highest first similarity. Finally, the code segment, sentence, or text block information related to the selected target embedding vector is used to extract the answer.

In this embodiment, each text block, each code segment, and each sentence are taken as a unit and are input into the model to obtain corresponding embedding vectors to construct an embedding vector set. Since the embedding vector captures the semantic information of the text, for the query statement raised by the user, it is only necessary to calculate the first similarity between the embedding vector of the query statement and the embedding vectors of each code segment, sentence, and text block in the embedding vector set, and find the target embedding vector with the highest first similarity, so as to quickly obtain the most relevant information and improve the efficiency of answering.

Based on any of the foregoing embodiments, obtaining answer information corresponding to the query statement according to the target embedding vector includes:

Obtaining a target text represented by the target embedding vector;

The target text is used as context information and is input into a third natural language model together with the query statement to obtain answer information corresponding to the query statement.

Among them, the target text represented by the target embedding vector is a specific text fragment selected when constructing the target embedding vector, which can be any one of a code segment, a sentence or a text block, or any one of a code comment corresponding to a code segment, a literature answer corresponding to a sentence or a text block.

In this embodiment, the target text is provided to the third natural language model as context information, and the user's query statement is input into the third natural language model as the current query question. The third natural language model will generate corresponding answer information based on the context information and the query question.

Among them, the third natural language model can be a model with the same network architecture as the first natural language model but different model parameters, or it can be a model with different network architecture and model parameters from the first natural language model, and there is no limitation on this.

In this embodiment, the target text represented by the target embedding vector with the highest first similarity to the embedding vector of the query statement is used as context information and input into a third-party natural language model together with the query statement. This can provide richer context and background information, thereby improving the accuracy, coherence and comprehension ability of the model, and obtaining better answer information.

Based on any of the above embodiments, determining the document cited by the project README file and obtaining a document file in a preset format corresponding to the document includes:

Matching the text information of the references in the project README file based on a preset regular expression;

Parsing meta information of the cited document based on the text information;

A text file in a preset format is obtained from an open source document library based on the meta information.

Meta-information of a document refers to the information that describes and defines the document, so as to facilitate storage, management, retrieval and use. Meta-information may include author name, document title, publication date, journal name, volume number, issue number, page number, DOI, abstract, keywords, etc.

In this embodiment, a regular expression method is used to determine the documents cited by the project self-described file. Specifically, an appropriate preset regular expression can be defined to match the text information according to the characteristics of the text information of the cited document. For example, a preset regular expression can be used to match meta-information such as document title, author, conference/journal name, year, etc.

The text information of the matched references is further parsed to obtain meta-information, such as document title, author, conference/journal name, year, etc. Regular expressions or other text analysis techniques can be used to extract this information.

For the parsed meta information, a query request can be constructed to obtain a text file in a preset format. The query request can include meta information such as document title, author, conference/journal name, and year. After the query request is constructed, the constructed query request is sent to the open source document library and the corresponding response result is obtained. Specifically, according to the API document or interface specification of the open source document library, the corresponding request and parsing processing are performed to obtain the required text file in the preset format.

The code parsing device provided by the present invention is described below. The code parsing device described below and the code parsing method described above can be referenced to each other.

Figure 2 is one of the structural schematic diagrams of the code parsing device provided by the present invention, as shown in Figure 2, including: a first code parsing module 210, used to determine the code files that need to be run in the project according to the project self-description file, divide the codes in the code files into fragments with functions as the smallest units, and obtain a plurality of code segments; a second code parsing module 220, used to determine the code hierarchy relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code files; a third code parsing module 230, used to input the plurality of code segments into the first natural language model in sequence according to the first preset instruction format in the order of code hierarchy from low to high, and obtain the code comments corresponding to each of the code segments, the first preset instruction format is used to instruct the first natural language model to generate the code comments of the code segments, and the first natural language model is obtained by training the initial natural language model according to the original code segments that identify the original code comments.

The code parsing device provided by the present invention first determines the code files that need to be run in the project according to the project self-description file, divides the codes in the code files into fragments with functions as the smallest units to obtain a plurality of code segments, and then determines the code hierarchy relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file, so as to build a bottom-up logical system through the code hierarchy relationship, and then inputs the plurality of code segments into the first natural language model in sequence according to the first preset instruction format in the order from low to high of the code hierarchy. By inputting the code segments into the first natural language model in the order from low to high of the code hierarchy, the first natural language model can better understand the context and logical flow of the code, so that the first natural language model can use the code hierarchy relationship between the code segments to generate more accurate annotations, thereby realizing rapid code interpretation.

In some embodiments, the code parsing device also includes: a fourth code parsing module, used to determine the literature cited by the project self-description file, and obtain a literature file in a preset format corresponding to the literature; divide the content in the literature file into a plurality of sentences according to a preset delimiter; input a plurality of the sentences into a second natural language model in turn according to a second preset instruction format to obtain a literature answer corresponding to each of the sentences, and the second preset instruction format is used to instruct the second natural language model to generate a literature answer corresponding to the sentence; based on the text similarity between the code annotation corresponding to the code segment and the literature answer corresponding to the sentence, establish an association between the code segment and the sentence to assist in code interpretation.

In some embodiments, the code parsing device also includes: a fifth code parsing module, which is used to perform block processing on the text information in the project self-description file to obtain a plurality of text blocks; obtaining an embedding vector set, wherein the embedding vector set includes an embedding vector of each of the code segments, an embedding vector of each of the sentences, and an embedding vector of each of the text blocks; upon receiving a user's query statement, obtaining the embedding vector of the query statement; based on a first similarity between the embedding vector of the query statement and each embedding vector in the embedding vector set, selecting a target embedding vector with the highest first similarity from the embedding vector set; and based on the target embedding vector, obtaining answer information corresponding to the query statement.

In some embodiments, the fifth code parsing module is also used to obtain the target text represented by the target embedding vector; use the target text as context information and input it into the third natural language model together with the query statement to obtain answer information corresponding to the query statement.

In some embodiments, the first code parsing module is also used to match the text information of the referenced documents in the project README based on a preset regular expression; parse the meta-information of the referenced documents based on the text information; and obtain a text file in a preset format from an open source document library based on the meta-information.

In some embodiments, the third code parsing module is also used to crawl the original code segment with the original code annotation; input the original code segment into the initial natural language model to obtain the predicted annotation output by the initial natural language model; according to the second similarity between the predicted annotation and the original code annotation, the model parameters of the initial natural language model are updated until the second similarity reaches the preset similarity requirement, thereby obtaining the trained first natural language model.

FIG3 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG3 , the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the code parsing method applied to the server device, the method comprising: determining the code file to be run in the project according to the project self-reading file, dividing the code in the code file into fragments with functions as the smallest unit, and obtaining a plurality of code segments; determining the code level relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file; and inputting the plurality of code segments into the first natural language model in sequence according to the first preset instruction format in the order from low to high code levels, and obtaining the code comments corresponding to each of the code segments, wherein the first preset instruction format is used to instruct the first natural language model to generate the code comments of the code segments, and the first natural language model is obtained by training the initial natural language model according to the original code segment with the original code comments identified.

In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the code parsing method provided by the above methods and applied to a server-side device, the method including: determining the code file that needs to be run in the project according to the project self-reading file, dividing the code in the code file into fragments with functions as the smallest unit, and obtaining a plurality of code segments; determining the code hierarchy relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file; and inputting the plurality of code segments into a first natural language model in sequence according to a first preset instruction format in order of code hierarchy from low to high, to obtain code comments corresponding to each of the code segments, wherein the first preset instruction format is used to instruct the first natural language model to generate code comments for the code segments, and the first natural language model is obtained by training an initial natural language model based on an original code segment that identifies the original code comments.

On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the code parsing method applied to a server-side device provided by the above-mentioned methods, the method comprising: determining the code file that needs to be run in the project according to the project README file, dividing the code in the code file into fragments with functions as the smallest unit, and obtaining a plurality of code segments; determining the code hierarchy relationship corresponding to the plurality of code segments according to the calling relationship of the functions in the code file; and inputting the plurality of code segments into a first natural language model in sequence according to a first preset instruction format in order of code hierarchy from low to high, and obtaining code comments corresponding to each of the code segments, wherein the first preset instruction format is used to instruct the first natural language model to generate code comments for the code segments, and the first natural language model is obtained by training an initial natural language model based on an original code segment that identifies the original code comments.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A method of code resolution, the method comprising:

Determining a code file to be operated in the project from the file according to the project, dividing codes in the code file into segments taking a function as a minimum unit, and obtaining a plurality of code segments;

determining the code hierarchy relation corresponding to the plurality of code segments according to the calling relation of the functions in the code file;

Sequentially inputting a plurality of code segments into a first natural language model according to a first preset instruction format according to the sequence of the code levels from low to high to obtain code annotations corresponding to the code segments, wherein the first preset instruction format is used for indicating the first natural language model to generate the code annotations of the code segments, and the first natural language model is obtained by training the initial natural language model according to the original code segments with the original code annotations.

2. The code parsing method of claim 1, wherein the method further comprises:

determining a document cited by the item from the document, and acquiring a document in a preset format corresponding to the document;

dividing the content in the document file into a plurality of sentences according to preset separators;

Sequentially inputting a plurality of sentences into a second natural language model according to a second preset instruction format to obtain literature solutions corresponding to the sentences, wherein the second preset instruction format is used for instructing the second natural language model to generate the literature solutions corresponding to the sentences;

based on the text similarity between the code annotation corresponding to the code segment and the document solution corresponding to the sentence, an association between the code segment and the sentence is established to assist in code interpretation.

3. The code parsing method of claim 2, wherein the method further comprises:

Carrying out blocking processing on text information in the project self-contained file to obtain a plurality of text blocks;

acquiring an embedded vector set, wherein the embedded vector set comprises embedded vectors of all the code segments, embedded vectors of all sentences and embedded vectors of all the text blocks;

Under the condition that an inquiry sentence of a user is received, acquiring an embedded vector of the inquiry sentence;

selecting a target embedded vector with highest first similarity from the embedded vector set according to the first similarity between the embedded vector of the query statement and each embedded vector in the embedded vector set;

and acquiring answer information corresponding to the query statement according to the target embedded vector.

4. The code parsing method according to claim 3, wherein the obtaining answer information corresponding to the query statement according to the target embedded vector includes:

Acquiring a target text represented by the target embedded vector;

And taking the target text as context information, and inputting the context information and the query sentence into a third natural language model to obtain answer information corresponding to the query sentence.

5. The code parsing method according to claim 2, wherein the determining the document cited by the item from the document and acquiring the document file in the preset format corresponding to the document includes:

matching text information of a reference document in the project self-describing file based on a preset regular expression;

Parsing meta information of the cited document based on the text information;

and acquiring a text file in a preset format from an open source document library based on the meta information.

6. The code parsing method of claim 1, wherein the first natural language model is trained by:

Crawling an original code segment with original code annotations;

Inputting the original code segment into an initial natural language model to obtain a prediction annotation output by the initial natural language model;

And updating the model parameters of the initial natural language model according to the second similarity between the prediction annotation and the original code annotation until the second similarity meets the preset similarity requirement, so as to obtain the trained first natural language model.

7. A code parsing apparatus, the apparatus comprising:

The first code analysis module is used for determining a code file to be operated in the project according to the project self-description file, dividing codes in the code file into segments taking functions as minimum units, and obtaining a plurality of code segments;

the second code analysis module is used for determining the code hierarchy relation corresponding to the plurality of code segments according to the calling relation of the functions in the code file;

The third code analysis module is used for sequentially inputting a plurality of code segments into a first natural language model according to a code level from low to high and a first preset instruction format to obtain code annotations corresponding to the code segments, the first preset instruction format is used for indicating the first natural language model to generate the code annotations of the code segments, and the first natural language model is obtained by training the initial natural language model according to the original code segments with the original code annotations.

8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the code resolution method of any one of claims 1 to 6 when the program is executed by the processor.

9. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the code parsing method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program which, when executed by a processor, implements the code parsing method according to any one of claims 1 to 6.