JP2025171397A - Computer program, information processing device, and information processing method - Google Patents

Abstract

A computer program, an information processing device, and an information processing method for supporting the creation of software development-related documents are provided. The computer program acquires document data of software development-related documents, provides the document data to a language model so as to output review results for the software development-related documents, and executes a process for outputting the review results based on the output from the language model. [Selected Figure] Figure 3

Description

The present invention relates to a computer program, information processing device, and information processing method that support the creation of software development-related documents.

Technology has been proposed for automatically checking the content of various business-related documents with defined formats by using OCR (Optical Character Recognition/Reader) or other tools to capture the text within the document as text data (see, for example, Patent Document 1).

JP 2024-006563 A

In cases where laws governing contracts, such as those disclosed in Patent Document 1, or software programs have defined grammar, it is possible to correct text captured by OCR based on a comparison with clear laws or grammar.

In contrast, while there are no clear national laws or globally-defined rules, there are documents whose formats, wording, and required information are relatively loosely defined based on the knowledge and customs accumulated by each company, organization, and technical field. For example, information that should be shared among project participants, such as "requirements specifications," "development specifications," and "design documents" that define the necessary functions in a system development project, has a certain degree of format to ensure there are no discrepancies in mutual understanding. If the checking and correction of this document data could be automated, it could contribute to improving work efficiency, etc.

The present invention aims to provide a computer program, information processing device, and information processing method that support the creation of software development-related documents.

A computer program according to one embodiment of the present disclosure causes a computer to acquire document data for software development-related documents, provide the document data to a language model so as to output review results for the software development-related documents, and execute a process of outputting the review results based on the output from the language model.

In the computer program, information processing device, and information processing method disclosed herein, review results for software development-related document data are automatically output using a language model. The language model may reference not only the software development-related document data being reviewed, but also reference reference information for the software development-related documents, the technical field of the software being developed, and accumulated knowledge to date, and output the review using RAG (Retrieval-Augmented Generation) technology.

This disclosure enables reviews of software development-related documents to be automatically output, enabling efficient checking of documents for items that need to be corrected.

FIG. 1 is a schematic diagram of a development support system. FIG. 1 is a block diagram showing a configuration of an information processing device. FIG. 2 is an explanatory diagram of a language model module. FIG. 2 is an explanatory diagram of knowledge data. FIG. 2 is a block diagram showing the configuration of a terminal device. 10 is a flowchart illustrating an example of a procedure for a review output process in the development support system. FIG. 1 is a diagram illustrating an example of a software development related document. FIG. 10 is a diagram illustrating an example of a software development-related document converted into text. FIG. 10 is a diagram illustrating an example of a prompt. FIG. 10 is a diagram showing an example of a display of review results. FIG. 10 is a schematic diagram of a language model module according to the second embodiment. 10 is a flowchart illustrating an example of a review output processing procedure according to the second embodiment. FIG. 11 is a diagram showing an example of a display of a review result in the second embodiment. 13 is a flowchart illustrating an example of a review output processing procedure according to the third embodiment. FIG. 10 is a diagram illustrating an example of a feedback selection interface displayed on a terminal device.

The present disclosure will be described in detail with reference to drawings showing embodiments thereof. In the following embodiment, a development support system using the computer program of the present disclosure will be described.

Figure 1 is a schematic diagram of a development support system 100. The development support system 100 includes an information processing device 1 and a terminal device 2. The information processing device 1 is a web server managed by a development support provider, and the terminal device 2 is capable of communicating with the information processing device 1 via a network N. The terminal device 2 is, for example, a personal computer used by an operator who designs and develops a large number of software programs related to the system.

On terminal device 2, the operator creates and edits software development-related documents such as "development specifications," "design documents," and "requirements definition documents" for the software being designed and developed. On terminal device 2, the operator performs operations to request a review (content check, suggested revisions, etc.) of the software development-related documents they have created and edited. Upon receiving the request from terminal device 2, information processing device 1 acquires the software development-related documents and outputs the review results for the target to terminal device 2. In the following explanation, information processing device 1 does not automatically create or automatically correct software development-related documents, but simply outputs the review results.

Software development documents such as "development specifications," "design documents," and "requirements" are often updated through the intervention of multiple operators in response to the cyclical cycle of software creation, debugging, testing, and deployment, based on a quality improvement method known as continuous integration. When software changes are made due to increased system requirements, regression testing is performed to determine whether the update will affect other software components. Because regression testing requires significant time and expense, reducing the cost and streamlining this process is particularly important for system development and maintenance. Automated generation of various tests, including regression testing, and software itself is also being promoted, and software development documents such as "development specifications," "design documents," and "requirements" play an important role in this process. There are methods for automatically generating test code from software development documents and for verifying their correspondence with software development documents. However, errors or ambiguities in software development documents can hinder the development and testing process and lead to defects. The larger the system being developed, the more software-related documentation will be created, with varying degrees of detail and interrelatedness, increasing the likelihood of defects occurring due to discrepancies and the labor costs of regression.

Furthermore, systems often include a GUI (Graphics User Interface), and software development-related documents often include specifications with images of the software's GUI. Software development-related documents often use diagrams and charts to enhance intuitive understanding among operators and developers. Not only GUIs, but even tables may be color-coded to facilitate intuitive understanding. Even when applying conventional OCR technology to such software-related documents, it is difficult to obtain the text information expressed within the images. Furthermore, documents that contain a mixture of image data and text data are difficult to handle directly using natural language processing.

In the development support system 100 disclosed herein, the information processing device 1 uses a language model LM to appropriately output at least review results for errors in software development-related documents based on the developer's knowledge and practices. Furthermore, by performing image recognition preprocessing on documents that contain images and converting them into text, GUI images within the documents can be included as a subject of review based on knowledge and practices using the language model LM. This means that operators of large-scale system developers can efficiently review software development-related documents and fix bugs, and can present review results such as suggested revisions based on accumulated knowledge. Operators can accurately and appropriately create and modify "development specifications" and "design documents." Even in today's world where there is a shortage of highly skilled personnel capable of such creation and modification, this system makes it possible to provide systems based on high-quality software suites.

The details of each device and process used to support the creation and revision of software development-related documents are described below.

Figure 2 is a block diagram showing the configuration of information processing device 1. Information processing device 1 is a server computer. The server computer may be a cloud server managed by a development support provider, or an on-premise server managed by a software development provider. Information processing device 1 may also be a desktop or laptop personal computer.

The information processing device 1 includes a processing unit 10, a memory unit 11, and a communication unit 12. The processing unit 10 is a processor such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc. The processing unit 10 includes memory such as SRAMs (Static Random Access Memory) and DRAMs (Dynamic Random Access Memory). The processing unit 10 reads the information processing program P1 stored in the memory unit 11 into memory and executes it, causing a general-purpose computer to perform various processes described below.

The memory unit 11 is a relatively large-capacity non-volatile storage medium (storage) such as a solid-state drive (SSD) or a hard disk drive. The memory unit 11 stores programs (program products) required for the processing unit 10 to execute processing, as well as reference setting data. The program products include an information processing program P1, a language model module M1, and knowledge data K1.

Any of the information processing program (computer product) P1, language model module M1, and knowledge data K1 stored in the memory unit 11 may be the information processing program P9, language model module M9, or knowledge data K9 stored on a computer-readable non-temporary storage medium 9 that has been read by the processing unit 10 and stored in the memory unit 11. Any of the information processing program P1, language model module M1, and knowledge data K1 may be downloaded by the processing unit 10 from an external download server via the communication unit 12 and stored in the memory unit 11.

Processing based on the information processing program P1, language model module M1, and knowledge data K1 will be described in detail below.

The communication unit 12 communicates with the terminal device 2 via the network N. The communication unit 12 is, for example, a network card. The processing unit 10 can receive data from the terminal device 2 and respond to the terminal device 2 via the communication unit 12.

Network N is a network that includes the so-called Internet. Network N may also be a local network of a software developer. Network N may also include a carrier network or other wireless communication media.

Figure 3 is an explanatory diagram of the language model module M1. The language model module M1 includes a model known as a language model (LM). The language model module M1 may also include a large language model (LLM) such as GPT-4. A portion of the language model module M1, such as the language model LM, may be running on an external server (GPT-4, Gemini, Cloude, etc.) and be used by the processing unit 10 via the communication unit 12. The language model LM is a model trained using a neural network to output natural language data (natural sentences) using words with a high occurrence probability for a group of words input as vector data. Various methods, including Transformer, can be used as the language model LM algorithm.

The language model module M1 uses the above-mentioned language model LM to input data for software development-related documents such as "development specifications" converted to text as described below, along with a query for that data, and outputs the review results for the software development-related documents in natural language. The language model module M1 either instructs the language model LM to output the data in a format that can be read by spreadsheet software (for example, separated by characters or symbols such as commas or colons), or formats and then outputs the data.

The language model module M1 may be used based on the knowledge data K1, combining knowledge about software development-related documents contained in the knowledge data K1. In other words, the information processing device 1 uses the language model LM with improved accuracy using RAG (Retrieval Augmented Generation) technology.

Knowledge data K1 is reference information prepared for software development-related documents. Knowledge data K1 may be prepared individually for each business field to which the target software is applied and for each piece of target software. Knowledge data K1 is data that can be used as reference information such as assumptions for the language model LM. Knowledge data K1 is, for example, document data of "development specifications," "design documents," and "requirements definition documents" that have been created, reviewed, and revised. Knowledge data K1 may be, for example, textual data of expert comments on software changes and regression testing, which have been converted into vector data and stored. Knowledge data K1 may also be expert comments on past documents. It is preferable that knowledge data K1 be converted into vector data for ease of search and for input into the language model LM, described below.

Knowledge data K1 may be stored for each business field according to the type of software development-related document, such as "development specifications," "design documents," and "requirements definition documents." Figure 4 is an explanatory diagram of knowledge data K1. In Figure 4, knowledge data K1 is stored so that the type of software development-related document and the corresponding business field can be identified. In the example of Figure 4, knowledge data K1 is stored separately for each business field, "ERP" and "production management." It may also be separated into other business fields, such as education-related fields. Knowledge data K1 is stored separately for each business field, with data on "development specifications," data on "design documents," and data on "requirements definition documents." Knowledge data K1 may also be stored so that it can distinguish between software development-related documents with varying levels of detail, from coarse data such as design documents that merely indicate the overall "overview" or "concept" of the system, to dense data such as the "individual knowledge" of each software module that implements the system.

Knowledge data K1 may contain overlapping data from different business fields or types. Knowledge data K1 may include data that can be referenced in any business field or type. By using such knowledge data K1, language model module M1 also functions as a means for understanding the content of input software development-related documents.

As described below, knowledge data K1 can be updated based on past review results (remarks made by experts). Knowledge data K1 can be learned and updated to encourage a method of making comments supported by operators at software development companies. By learning and updating this knowledge data K1, it is possible to improve the accuracy of review results by utilizing knowledge accumulated by development support companies while using a language model LM used via an external server.

Returning to Figure 3, we will further explain the language model module M1. The language model module M1 is given a query corresponding to a request to the language model LM and text data of software development-related documents. The language model module M1 interprets the given query and text data, selects the necessary knowledge data K1, and creates a prompt for the language model LM.

Language model module M1 also includes modules that perform computational processes such as tokenization and embedding of input query and software development-related document data. Language model module M1 also includes a module (Concatenate) that concatenates the processed data of the given query and software development-related document with elements extracted from the vectorized knowledge data K1. Language model module M1 inputs the concatenated data (vector data) into a language model (large-scale language model).

The data extracted from the knowledge data K1 may be given to the language model LM as a premise before or after inputting the processed data of the input software development-related documents, so that the output language data relates to the software development-related documents.

Knowledge data K1 may be product information for the software that is the subject of the software-related document, or information about its business field. If the subject software is related to ERP (Enterprise Resources Planning) or a core system, knowledge data K1 should include terminology definitions and scale information for the system corresponding to each of multiple business operations. This allows language model module M1 to output language-type data for the input software development-related document, such as changes to the software development-related document and comments from experienced developers of the subject software or experienced regression testers, based on knowledge data K1.

The language model LM may be tuned in advance using instruction data that includes the professional role and comment format of the development support provider to make changes to software development-related documents such as specifications and to display the comments made by an expert in regression testing.

Knowledge data K1 may be stored in the form of a data set of, for example, "development specification" data (input) including past errors and areas to be corrected, as knowledge about the "development specification," and review results (output) by an operator with advanced knowledge of the "development specification." In this case, language model module M1 is not limited to a so-called language model (LM), and may be trained using various natural language analysis algorithms so that the comments made by an operator with advanced knowledge of the "development specification" can be reproduced for the input "development specification" data. The configuration of language model module M1 is not limited to the form shown in FIG. 3.

Figure 5 is a block diagram showing the configuration of terminal device 2. Terminal device 2 is a laptop or desktop personal computer used by an operator of a software developer. Terminal device 2 may also be a smartphone or tablet terminal.

The terminal device 2 includes a processing unit 20, a memory unit 21, a communication unit 22, a display unit 23, and an operation unit 24.

The processing unit 20 is one or more processors such as a CPU, MPU, GPU, etc. The processing unit 10 includes memory such as SRAM and DRAM. The processing unit 20 controls each component to execute processing. The processing unit 20 may be configured as a single piece of hardware (SoC) that integrates a processor, memory, and a communication unit 22 (described below). The processing unit 20 sends and receives data to and from the information processing device 1 based on a program stored in the storage unit 21, and accepts operations from a test engineer. Details will be provided below.

The storage unit 21 uses non-volatile memory such as a hard disk, flash memory, or SSD. The storage unit 21 stores programs and data referenced by the processing unit 20. The storage unit 21 stores a terminal program (program product) P2. The terminal program P2 is a web browser program, and may be a web application program provided by the information processing device 1. The terminal program P2 stored in the storage unit 21 may be a terminal program P8 stored in the storage medium 8 that the processing unit 20 reads, copies, and installs. The terminal program P2 may also be a program downloaded by the terminal device 2 from a program server (not shown), stored in the storage unit 21, and installed.

The communication unit 22 is a communication device that realizes communication via the network N. The communication unit 22 is, for example, a network card compatible with the network N. The communication unit 22 may also realize wireless communication. The processing unit 20 can send and receive data to and from the information processing device 1 via the communication unit 22.

The display unit 23 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 23 may also be a display with a built-in touch panel. Information may be output using an audio output unit including a speaker.

The operation unit 24 is a user interface that allows input and output to and from the processing unit 20. The operation unit 24 may be a touch panel built into the display unit 23, or may include physical buttons, switches, and physical dials. The operation unit 24 may also be an audio input/output unit that includes a microphone and speaker.

The processing in the development support system 100 configured in this manner will be described with reference to a flowchart. Figure 6 is a flowchart showing an example of the procedure for review output processing in the development support system 100. When a specific operation is performed on a software development-related document for a system development project on the terminal device 2 used by an operator, the processing unit 20 starts the following processing based on the terminal program P2.

A specific operation is, for example, an operation in which a user right-clicks the pointing device of the operation unit 24 on the icon of a data file of a software development-related document displayed on the display unit 23, thereby displaying a list of selectable processes for that data file, and then selecting a terminal program P2 included in that list. A specific operation may also be an operation in which the user selects the target data file on an operation panel including a GUI based on the terminal program P2, or dragging and dropping the target data file onto the operation panel.

Based on the terminal program P2, the processing unit 20 of the terminal device 2 acquires the data file (document data) that is the target of a specific operation (step S201). In step S201, the processing unit 20 expands the contents (document data) of the target data file into memory.

The processing unit 20 converts the contents of the data file into text data (step S202). The conversion to text data in step S202 is a process performed so that the language model LM included in the language model module M1 can understand the contents of documents related to software development. For this reason, the processing unit 20 may convert to text data according to predetermined rules using specific symbols, as described below in Figures 7 and 8. In step S202, if the processing unit 20 can obtain tables as text from the contents of the data file (document data), for example, it will convert these into text data in addition to the text included in the contents of the data file. In step S202, the processing unit 20 may also obtain the file name of the data file as text data.

The processing unit 20 extracts and formats text data from the image data of figures and tables included in the document data (step S203). In step S203, the processing unit 20 formats the extracted text data so that it is easy for the language model LM to understand that it is text explaining a figure or table. For example, the processing unit 20 may extract text data such as "data input" and "item" from the image data of a GUI screen and format it into natural language such as "There is a "data input" column. You can select the "item" of the data to input." In step S203, the processing unit 20 may extract text included in the image data using image recognition technology or OCR. In step S203, the processing unit 20 may use a neural network to convert the image data of a GUI screen into text data using a learning model that has been trained to output an overview of the GUI screen as text when the image data is input. In step S203, the processing unit 20 may use an existing image recognition model using technologies such as Vision Transformer or LayoutLM to recognize text contained in image data of diagrams and tables included in the contents of software development-related documents, along with the position within the image data where the text is contained (e.g., which line, etc.).

The processing unit 20 specifies the text data obtained in step S202 and the text data obtained in step S203 and sends a review request to the information processing device 1 (step S204).

The processing unit 10 of the information processing device 1 receives a review request specifying text data from a software development-related document (step S101). Step S101 enables the processing unit 10 to obtain text data converted or extracted from the software development-related document.

The processing unit 10 selects knowledge about the software development-related document that is the subject of the review request from the knowledge data K1 (step S102). In step S102, for example, if the subject software development-related document is a "development specification," the processing unit 10 selects knowledge about the "development specification" from the knowledge data K1, and if the subject software development-related document is a "requirements definition document," the processing unit 10 selects knowledge about the "requirements definition document" from the knowledge data K1. Note that the processing of step S102 may be performed by having the language model LM search during the processing of S104 described below.

The processing unit 10 creates a prompt for the language model module M1 (step S103). In step S103, the processing unit 10 creates text data as the prompt, which includes the role of the language model LM and instructions that specify the output format, etc. for the language model LM.

The processing unit 10 provides the text data of the software development-related document received in step S101, the data corresponding to the knowledge selected in step S102, and the text data of the prompt created in step S103 to the language model module M1 (step S104). In step S104, the processing unit 10 converts the prompt written in natural language and the text data of the software development-related document into vector data using the language model module M1, connects the data of the selected knowledge data K1, and provides the combined input data to the language model LM. In steps S103 and S104, the processing unit 10 inputs the prompt to the language model module M1 in a format that specifies the target text data and the knowledge data K1 to be referenced, and the language model module M1 and the language model LM may themselves reference the text data and knowledge data K1.

The processing unit 10 stores the review results output from the language model module M1 by the processing of step S104 in a temporary storage medium or the memory unit 11 (step S105). When storing the review results in the memory unit 11 in step S105, the processing unit 10 may store the review results in association with data identifying the target software development-related document and identification data identifying the review request (request ID, identification data of the terminal device 2 that made the request, ID of the operator of the terminal device 2, etc.).

In step S105, the processing unit 10 selects knowledge from the knowledge data K1, and if there is an indication that the knowledge is "incomprehensible" despite the instructions given in the query, it stores the fact that the knowledge is "incomprehensible" in the review results.

In step S105, the processing unit 10 associates each review result with its validity and stores it as knowledge data K1 in the memory unit 11, which can then be used to improve the accuracy of output by the language model module M1. The validity of the review results may be evaluated by a professional at the development support provider, or by an operator based on customer satisfaction.

The processing unit 10 converts the review results output from the language model module M1 into a predetermined format (for example, a file in the format of a specific spreadsheet software) (step S106) and transmits the converted results to the terminal device 2 (step S107). In step S106, the processing unit 10 converts, for example, CSV-format data output from the language model LM into data in a format for a specific application program. The conversion in step S106 may also be performed within the language model module M1 using a directive included in the language model module M1 that specifies the output format for the language model LM.

The processing unit 20 of the terminal device 2 receives the review results (step S205) and stores them in the storage unit 21 as a data file of the review results, giving it a name associated with the file name of the data file acquired in step S201 (step S206), and then terminates the process.

In the processing procedure shown in Figure 6, text data is extracted from GUI capture images and other data contained in the software development-related documents and transmitted (S203). However, step S203 can be omitted for software development-related documents that do not contain image data. In other words, the above processing procedure can of course also be applied to text-based software development-related documents that do not contain image data.

The above processing procedure will be explained using a specific example. Figure 7 is a diagram showing an example of a software development-related document. Figure 7 shows a "development specification" as a specific example. A "development specification" is document data that is primarily text. As shown in Figure 7, a "development specification" specifies a "number" that identifies the specification, the "process name" that the specification targets, the purpose of the processing, and so on. The example of a "development specification" shown in Figure 7 makes extensive use of a table format to make the explanation of each item easier to understand intuitively.

In the example of the "Development Specifications" shown in Figure 7, the section "2. Screen Layout and Item Descriptions" includes a captured image of a GUI screen. In the example of Figure 7, the captured image is highlighted with bold lines to highlight the parts that correspond to the processing to be used. In software development-related documents, not just "Development Specifications," this captured image may also contain handwritten or font-based text.

In the example "Development Specifications" shown in Figure 7, the layout of the GUI screen defines the upper section, including functions and menus such as "Exit," "Item Selection," "Settings," and "Copy," as the "Top of the Screen," the middle section including the drop-down list and input fields as the "Condition Section," and the table section at the bottom as the "Details Section." GUI screens are used by selecting the data item to be entered from the drop-down list and entering data in the input fields. Note that the GUI screen does not contain any text defining these "Top of the Screen," "Condition Section," or "Details Section," and these names are used as definitions of the "screen items" within the document. To understand the correspondence with each section of the GUI screen, readers of the "Development Specifications" require knowledge (prerequisites and reading comprehension).

GUI screens include input fields, as shown in Figure 7. In the case of the "Development Specifications" in Figure 7, an input field is provided in the part of the "Condition Section" in the middle of the GUI screen that is surrounded by a bold line, and this part is defined as "Data Entry." Users of the software being developed can enter text under specified rules. In the example "Development Specifications" shown in Figure 7, the screen item part of the "Condition Section" input field called "Data Entry" has the following explanation: "When data is entered, the system will automatically begin processing." Furthermore, in the "Development Specifications" in the example, for the screen item part of the "Condition Section" input field called "Data Entry," "Specified Month" is defined as the "Range" to be entered in the input field, and the "Explanation" field has the explanation: "Specify the data range."

Figure 8 shows an example of a software development-related document that has been converted into text. Figure 8 shows a portion of the "Development Specifications" shown in Figure 7 that has been converted into text. As shown in Figure 8, the converted data is expressed as text only. In the example of Figure 8, the position of each piece of text in the "Development Specifications" is indicated by a line number. The line number may not be indicated by a number, but the position may be expressed by the line number within the text data shown in Figure 8.

As mentioned above, the "Development Specifications" shown in Figure 7 makes extensive use of diagrams and charts to make the explanations for each item easier to understand intuitively. In contrast, the example shown in Figure 8 shows a "Table" that has been converted to text. In the example shown in Figure 8, the converted document (text data) encloses the names of the items in the "Table" in specific symbols (here, double quotation marks "") and converts the elements of the "Table" into strings using curly brackets { } to indicate them as matrices. Expressions using these symbols to identify the location of text within a table or diagram follow rules defined by the terminal program P2 and the information processing device 1 that receives it. The example shown in Figure 8 uses the JSON format, with curly brackets { } indicating the range of text included in the table and each piece of text within the table being represented line by line. Table items are indicated by double quotation marks, and the data for each item is indicated by the ":" symbol. Depending on the format of the data file for the original "Development Specifications," it is preferable to format it so that it can be more accurately understood by the information processing device 1 and language model LM.

In the example shown in Figure 8, for the "Table" in "2. Screen Layout and Item Description" in the "Development Specifications" shown in Figure 7, the row for the "Data Entry" item has "Specified Month" set in the "Range" field in the column direction, and the text indicates that "When data is entered, the system will automatically begin processing" and "Specify the data range" has been entered in "Explanation."

In the example shown in Figure 8, the image data included at the beginning of "2. Screen Layout and Item Description" in the "Development Specifications" shown in Figure 7 has been converted to text with the identification data "Image 1" for identifying the image data enclosed in curly brackets. It is possible to recognize from the text that the image data assigned the identification data "Image 1" will be displayed in this location.

When the text shown in Figure 8 and the image data corresponding to the text are passed to the information processing device 1, the processing unit 10 selects knowledge from the knowledge data K1 related to the "development specifications" as shown in Figures 3 and 6, and provides this knowledge to the language model module M1 along with the passed text. At this time, the processing unit 10 may create a prompt for the language model LM, as described in step S103 of Figure 6.

9 is a diagram showing an example of a prompt. The processing unit 10 generates text in natural language describing the roles "software verification expert," "role in ensuring software quality," and "prerequisites" such as "familiarity with ERP" corresponding to the language model LM. Adding such prerequisites can improve the accuracy of the review results. Furthermore, the processing unit 10 generates text in natural language describing "instructions" to the language model LM, such as that the review target is "software development specifications," that "please point out any ambiguities in the specifications," that "please provide the original text of the pointed-out parts, the details of the points, and proposed revisions," and that "please consider that the software is an ERP product" and "please output in a format compatible with spreadsheet software," corresponding to the "development specifications" being reviewed. Prompts composed of such natural language text can improve the accuracy of the review results output from the language model LM. For example, the instruction "please consider that the software is an ERP product" allows the language model LM to use not only the provided knowledge data K1 but also other searched results, which is expected to improve the accuracy of the review results.

Figure 10 shows an example of the display of review results. As shown in Figure 10, the review results include the position of the pointed-out passage in the original document (in this case, the line number), the content of the original text, the points pointed out by language model module M1, and suggested corrections for the pointed-out passage, all expressed in a format that can be read by spreadsheet software (e.g., CSV data) in accordance with the query shown in Figure 9.

In the review results shown in Figure 10, the ambiguity of the "explanation" text in the table in the "Development Specifications" in the example of Figure 7, i.e., the meaning cannot be uniquely identified (meaning it is incomprehensible), is also output in correspondence with the pointed out points. Similarly, in the review results shown in Figure 10, the unclear format of the "Specified Month" item that should be specified in the table in the "Development Specifications" in the example of Figure 7 is output in correspondence with the pointed out points.

An operator creating or updating software development-related documents specifies a "development specification" as shown in Figure 7 and executes processing using terminal program P2. This causes the terminal device 2 to convert the text shown in Figure 8 and send a review request specifying the converted text to the information processing device 1. At this time, based on terminal program P2, the processing unit 20 adds the data file name, i.e., the document type "development specification," to the review request and sends it to the information processing device 1, making it easier for the information processing device 1 to identify the knowledge in knowledge data K1.

Obtaining review results like those shown in Figure 10 allows operators who create or update software development-related documents to understand what needs to be corrected in the documents they create. For example, by displaying these tabular review results on the display unit 23 using spreadsheet software, the operator can recognize the specified items, including suggested corrections. This means that the operator does not need to read through or carefully read the document multiple times. Furthermore, these review results reflect the knowledge of engineers who are familiar with software development-related documents. This means that operators can avoid making unnecessary corrections to documents that could cause discrepancies with other specifications.

The above-described processing procedure and the resulting review result data enable operators of large-scale system developers to efficiently review software development-related documents and fix bugs. By assigning the role of the software developer to the large-scale language model (LLM), not only can a prompt to request revisions to the specifications be issued and a revision proposal be obtained, but also by reflecting knowledge data K1, it becomes possible to provide a more accurate and useful review for the operator. This makes it possible to provide systems with high-quality software even in situations where there is a shortage of highly skilled personnel.

Second Embodiment
In the second embodiment, a multimodal model that can input both text and image data is used. The configuration of the development support system 100 of the second embodiment is the same as the configuration of the development support system 100 of the first embodiment, except for the language model module M1 and detailed processing procedures described below. Therefore, among the configuration of the development support system 100 of the second embodiment, components that are common to the development support system 100 of the first embodiment are assigned the same reference numerals and detailed descriptions thereof will be omitted.

Figure 11 is a schematic diagram of the language model module M1 in the second embodiment. The language model module M1 includes a language model LM, as in the first embodiment. In the second embodiment, the language model LM may also be a large-scale language model, or may be one running on an external server that the processing unit 10 uses via the communication unit 12.

The language model module M1 of the second embodiment uses, for example, LLaVA (Large Language-and Vision Assistant). The language model module M1 is not limited to LLaVA, and other architecture models can be used. In the example shown in FIG. 11, the language model module M1 includes a module that performs computational processing on input text, such as tokenization and vector data embedding, similar to the language model module M1 of the first embodiment shown in FIG. 3. In the second embodiment, the language model module M1 vector-converts the input image data, encodes (abstracts) the vector-converted data, and further performs processing (languagezation) to identify noteworthy data using a Vision Transformer or the like. The language model module M1 may also use a model such as a Vision Transformer that outputs text describing what appears in the input image. The language model module M1 is capable of concatenating data after processing the text and data after processing the image data.

In the second embodiment, the language model module M1 is also used to reference knowledge about software development-related documents based on the knowledge data K1. The language model module M1 concatenates the vector data of the text of the software development-related documents, the vector data after processing the image data, data selected from the knowledge data K1, and the text data of the prompt, and provides this as input data to the language model LM. When processing image data, the language model module M1 may associate the relationships and structure between text, such as whether the text is a title or content, or whether the text is a major, medium, or minor item, with the software development-related documents and represent this data in a way that the language model LM can understand, depending on the position of the text within the image.

The types and usage of knowledge data K1 are the same as in the first embodiment, so detailed explanation will be omitted.

Figure 12 is a flowchart showing an example of the review output processing procedure in the second embodiment. In the second embodiment, when a specific operation is performed on a software development-related document on the terminal device 2, the processing unit 20 starts the following processing based on the terminal program P2.

The processing unit 20 of the terminal device 2 acquires the data file (document data) that is the target of a specific operation (step S211) and converts the contents of the data file into text data (step S212). In step S212, the processing unit 20 extracts the text data from the contents of the data file; it is not necessary to read any charts, tables, or images.

The processing unit 20 acquires image data and the position of the image from the contents of the data file (step S213). In step S213, the processing unit 20 may acquire diagrams and image data embedded in the data file, or may acquire image data of the screen (captured image) when the data file is displayed on the display unit 23. The processing unit 20 acquires the page number, line number, or coordinates as the position of the image. In step S212, when converting to text data, the processing unit 20 may write image identification data in text at the location where the image was included, and associate the identification data with the image data.

The processing unit 20 specifies the text data obtained in step S212 and the image data and position data obtained in step S213 and sends a review request to the information processing device 1 (step S214).

The processing of steps S212 and S213 may be performed by the information processing device 1. In this case, in step S214, the processing unit 20 transmits the acquired data file as is to the information processing device 1 together with a review request.

The processing unit 10 of the information processing device 1 receives a review request specifying text data, image data, and position data from a software development-related document (step S111). Step S111 enables the processing unit 10 to obtain text from the software development-related document.

The processing unit 10 selects knowledge about the software development-related document that is the subject of the review request from the knowledge data K1 (step S112). The processing content of step S112 is the same as the processing of step S102 in the processing procedure shown in Figure 6.

The processing unit 10 creates a prompt for the language model module M1 (step S113). The processing content of step S113 is the same as the processing content of step S103 in the processing procedure shown in Figure 6.

The processing unit 10 provides the language model module M1 with the text data of the software development-related document received in step S111, the image data and position data, the data corresponding to the knowledge selected in step S112, and the text data of the prompt created in step S113 (step S114). The processing of step S114 is similar to step S104 in FIG. 6. In step S114, the processing unit 10 may use the position data to identify text before and after the image, create a group of words with appearance probabilities (notable data) using the vector-converted image data and the identified text, combine this with the data in the knowledge data K1, and then provide it to the language model LM. In step S114, the processing unit 10 may create text indicating the position of the text in the image from the terminal device 2, including the instruction. For example, the processing unit 10 may convert the presence of specific text at a specific position in the image into text, and add a query that includes the text to point out any ambiguity.

The processing unit 10 stores the review results output from the language model module M1 by the processing of step S114 in a temporary storage medium or the storage unit 11 (step S115). The processing content of step S115 is the same as the processing content of step S105 in the processing procedure shown in Figure 6.

The processing unit 10 converts the review results output from the language model module M1 into a predetermined format (step S116) and transmits the converted results to the terminal device 2 (step S117). The conversion in step S116 may also be performed within the language model module M1 by specifying an output format for the language model LM included in the language model module M1.

The processing unit 20 of the terminal device 2 receives the review results (step S215) and stores them in the storage unit 21 as a data file of the review results, giving it a name associated with the file name of the data file acquired in step S201 (step S216), and then terminates the process.

In this way, the language model module M1 of the information processing device 1 can output a review that includes image data, without processing the diagrams or captured images contained in software development-related documents on the terminal device 2 side.

In the second embodiment, a multimodal model is used to obtain a review of the text and images of the software development-related document to be reviewed along with elements of knowledge data K1, making it relatively easy to obtain images as review results. Therefore, the processing unit 20 may output the review using diagrams.

Figure 13 shows an example of the display of review results in the second embodiment. If the review results relate to a GUI screen, the review results may be output as an image with a circle or other object superimposed on the portion of the GUI screen capture corresponding to the identified problem, as shown in Figure 13. The example in Figure 13, similar to Figure 10, shows a case in which text data and image data extracted from the "Development Specifications" data file shown in Figure 7 are sent directly to the information processing device 1, and the image data is also interpreted by the language model LM. By providing the language model LM with the "Development Specifications" field, the ERP industry field, and specific knowledge of the target software, the language model LM can recognize that the image data is a GUI screen and the correspondence between terms in each location on the GUI screen and other text. Therefore, the language model LM can, for example, identify variations in the spelling of the item name "Product Name" in the GUI screen and the "Item Name" described in the "Development Specifications." In the example of Figure 13, the review results are output by associating the spelling variations of the item name "Product Name" on the GUI screen with its position on the captured image of the GUI screen. By drawing and outputting an object that surrounds the item name "Product Name" on the GUI screen of the input image data, the language model LM can also output review results including findings about the captured image by associating them with its position on the captured image, as shown in Figure 12.

(Third embodiment)
In the third embodiment, the information processing device 1 tunes the language model module M1 itself or the knowledge data K1 based on feedback from an operator of software development-related documents, etc. The configuration of the development support system 100 of the third embodiment is the same as the configuration of the development support system 100 of the first embodiment, except for the language model module M1 and detailed processing procedures described below. Therefore, among the configuration of the development support system 100 of the third embodiment, components that are common to the development support system 100 of the first embodiment are assigned the same reference numerals and detailed description thereof will be omitted.

In order to efficiently receive feedback from operators, the development support system 100 of the third embodiment receives review requests from the terminal device 2 and outputs reviews to operators from the information processing device 1 via a web application. To this end, the information processing device 1 of the third embodiment stores a web server program for functioning as a web server and screen data for the web application in the storage unit 11, and the terminal program P2 of the terminal device 2 is a web browser program.

The process described below may not be a web application, but may involve sending and receiving data files of software development-related documents, files of review results for those documents, and feedback on the review results, as in the first or second embodiment.

Figure 14 is a flowchart showing an example of the review output processing procedure in the third embodiment. To obtain reviews of software development-related documents for a system development project, an operator launches terminal program P2 on the terminal device 2 and logs in with the operator's account, executing the following process.

Based on the terminal program P2, the processing unit 20 of the terminal device 2 acquires the software development-related document specified by the operator (step S221). The processing unit 20 then specifies the acquired software development-related document and sends a review request to the information processing device 1 (step S222).

The processing unit 10 of the information processing device 1 receives a review request specifying a software development-related document (step S121). The processing unit 10 converts the contents of the specified software development-related document into text data (step S122), extracts text data from images such as GUI screens included in the contents of the data file, and formats the text data (step S123). The processing of steps S122 and S123 is similar to the processing of steps S202 and S203 shown in Figure 6 of the first embodiment, and therefore detailed description will be omitted.

The processing unit 10 selects knowledge about the software development-related document that is the subject of the review request from the knowledge data K1 (step S124). The processing of step S124 is similar to the processing of step S102 shown in Figure 6.

The processing unit 10 creates a prompt for the language model module M1 (step S125). In step S125, the processing unit 10 creates text data as the prompt, including the role of the language model LM and instructions that specify the output format, etc. for the language model LM.

The processing unit 10 provides the text data of the software development-related document converted in step S122, the text data extracted from the image data in step S123, the data corresponding to the knowledge selected in step S124, and the text data of the prompt created in step S125 to the language model module M1 (step S126). In step S126, similar to the processing based on the language model module M1 described in the first embodiment, the processing unit 10 converts the text data into vector data, combines it with the data (vector data) selected from the knowledge data K1, and provides the combined input data to the language model LM along with an instruction to output the review.

The processing unit 10 stores the review results output from the language model module M1 by the processing of step S126 in association with identification data that identifies the review request (request ID, identification data of the terminal device 2 that made the request, ID of the operator of the terminal device 2, etc.) (step S127).

The processing unit 10 transmits screen data including the review results to the terminal device 2 (step S128). In step S128, the processing unit 10 transmits screen data including a portion showing the review results in tabular form and a selection interface for feedback on the review results (see Figure 15).

The processing unit 20 of the terminal device 2 receives the screen data in response to the review request (step S223). The processing unit 20 displays the screen on the display unit 23 based on the screen data (step S224) and accepts feedback on the review result on the screen (step S225). As described below, feedback can be selected from options such as "Correct," "Incorrect," or "Don't know." The processing unit 20 associates the accepted feedback with a review request ID or other identification data and transmits it to the information processing device 1 (step S226).

The processing unit 10 of the information processing device 1 receives the feedback (step S129), provides it as a reward to the language model LM of the language model module M1 (step S130), and terminates the process. This allows the language model LM of the language model module M1 to tune its policy based on the reward and output the next review with greater accuracy. In step S130, the processing unit 10 may update the knowledge data K1 using the vector data (after merging) of the target software development-related document, the review results, and the feedback.

Figure 15 is a diagram showing an example of a feedback selection interface 232 displayed on the terminal device 2. Figure 15 shows an example of a display screen 230 of review results displayed on the terminal device 2. As shown in Figure 15, the display screen 230 displays the review results 231 in table format, showing the position (line) of the pointed-out section within the document, the original text or diagram content (original text) of the pointed-out section, the pointed-out content, and suggested revisions. In the third embodiment, the display screen 230 includes a selection interface 232 corresponding to feedback options. As shown in Figure 15, the selection interface 232 includes options such as "useful" (correct point), "not useful" (incorrect point), "easy to understand" (correct point), and "difficult to understand" (unclear). When the operator selects one of the options, the selected content is sent to the information processing device 1 as feedback.

The above-described processing procedure and the resulting review result data enable the review results output from the information processing device 1 to be more accurate and satisfactory to the operator. It is preferable to update not only the language model module M1, including the language model LM, but also the knowledge data K1 based on operator feedback. By updating the knowledge data K1 and outputting accurate review results while maintaining the functionality of the language model LM, and tuning the knowledge data K1, it becomes possible to output reviews that sound like expert comments. The processing unit 10 may store and update pairs of input software development-related documents and output review results as knowledge data K1, or may store pairs of input software development-related documents, review results, and final software development-related documents after correction. Feedback from the operator of the software development-related documents regarding the review results may also be stored as knowledge data K1. This knowledge data K1 is expected to further improve the efficiency of software development document review and bug correction by operators of large-scale system developers. This makes it possible to provide systems with high-quality software suites, even in situations where there is a shortage of highly skilled personnel.

In the first and second embodiments described above, the information processing device 1 and the terminal device 2 are distinguished, and the operator uses the terminal device 2 to make a review request to the information processing device 1. The development support system 100 is not limited to the above-mentioned configuration, and the operator may use the information processing device 1 by connecting a display unit and an operation unit, and data transmission and reception between the terminal device 2 and the information processing device 1 may not be performed.

The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications that are equivalent in meaning to and within the scope of the claims.

1 Information processing device 10 Processing unit 11 Storage unit P1 Information processing program (computer program)
LM: Language model K1: Knowledge data 2: Terminal device 23: Display unit 24: Operation unit P2: Terminal program

Claims (13)

On the computer,
Acquire document data related to software development,
providing the document data to a language model so as to cause the language model to output a review result regarding the software development-related document;
A computer program that executes a process of outputting a review result based on the output from the language model. The computer,
2. The computer program according to claim 1, wherein the computer program executes a process of outputting the review results in a predetermined format including pointed out portions in the software development-related documents, pointed out matters for the pointed out portions, and suggested modifications for the pointed out portions. The computer,
creating a prompt including a role and instructions for the language model;
The computer program according to claim 1 , further comprising: providing the generated prompt together with the document data to the language model. The computer,
selecting knowledge data relating to the software development related documents;
The computer program according to claim 1 or 3, which causes the computer to execute a process of providing the selected knowledge data together with the document data to the language model. The computer program according to claim 4 , wherein the knowledge data is reference information prepared for each target software or field of the software development related document. The knowledge data is stored according to a plurality of document types for each business field of software development related documents,
The computer,
Identifying the business field and document type of the software development-related documents;
5. The computer program according to claim 4, wherein the computer program executes a process of outputting a review result by providing knowledge data corresponding to the identified business field and document to the language model together with the document data. the document data includes a captured image of a diagram or a GUI screen;
The computer,
Extracting text from the diagram or the captured image;
The computer program according to claim 1 , further comprising: providing the extracted text together with text included in the document data to a language model. The computer,
creating text data including extracted text and data indicating the position of the text in the diagram or the captured image;
providing the created text data to the language model;
The computer program according to claim 7 , which causes the computer to execute a process of outputting a review result including findings on the diagram or the captured image in association with a position within the diagram or the captured image based on an output from the language model. The computer,
The computer program according to claim 8 , which causes the computer to execute a process of outputting image data in which information specifying an issue pointed out about the diagram or the captured image is superimposed on the diagram or the captured image, the image data being included in the review result. The computer,
Accept evaluations of the output review results,
The computer program product of claim 1 , further comprising: training the language model based on received evaluations. The computer,
The computer program according to claim 4 , wherein the computer program executes a process of storing review results outputted for the software development related documents as the knowledge data. Acquire document data related to software development,
providing the document data to a language model so as to cause the language model to output a review result regarding the software development-related document;
An information processing device comprising: a processing unit that executes a process of outputting a review result based on an output from the language model. Acquire document data related to software development,
providing the document data to a language model so as to cause the language model to output a review result regarding the software development-related document;
and outputting a review result based on the output from the language model.