JP7703197B1 - Automatic programming function that reflects the results of checking input and output in program generation - Google Patents

Abstract

[Problem] In systems with automatic programming functions, the code generated by the automatic programming function for input text is often different from what was intended by the input text. Therefore, when using the automatic programming function, it is necessary for a person to execute the generated results, check them, and change the input to bring them closer to the originally intended program. This makes it possible to generate the intended program without human intervention.
[Solution] Using patterns that can exist as input derived from an input sentence and patterns that can exist as output for that input, the input patterns are given as input to the program and the equivalence of the resulting output is confirmed, and the accuracy of the program is achieved by disabling branch conditions that cannot be confirmed for equivalence.
[Selected Figure] Figure 1

Description

This invention relates to an automatic program generation function that automatically creates software, so-called automatic programming, and to an environment for executing automatic programming, i.e. technology that is effective when applied to the automatic program generation function.

When generative AI becomes more widely used and automatic program generation functions that do not require human programming, so-called automatic programming, become mainstream, it is believed that programs will be generated by artificial intelligence using natural language processing, and human programming will decrease. Currently, automatic programming generates programs by using natural language created by humans and associated with code, and programming code for natural language is associated in a database associated with the automatic program generation function or in a program with an automatic program generation function. Corresponding code is added to new words and sentences. The destination of insertion of this code is in a program with an automatic program generation function or in a commonly used database. As a result, regardless of the natural language input by each user, the result is common learning and growth, and all users who use it can use the relationship between the learned code and natural language. This structure with natural language corresponding code in the database uses the same database by sequentially synchronizing even in multiple databases. In addition, the structure with natural language corresponding code in the automatic program generation function can maintain the same automatic program generator by adding the learned results to all automatic program generation functions, recompiling, and sequentially synchronizing. This allows all users to enjoy the learned results in the same way. On the other hand, it is often the case that the resulting programs are unintended, and it is believed that whether or not the intended programs can be generated is important for the generalization of automatic programming.

Japanese Patent Application No. 09-214102: An automatic programming device having a code string correction means, which, for an irregular operand for a machine code generated by combining existing codes, removes the operand or changes the code for that operand Patent application 2023-010303: A test generation method in deep learning for improving the accuracy of pre-created program code JP 09-073306: A method for inputting data into a database that is closely related to program input, by making the input method for creating a program closer to natural language (several types are proposed) JP 06-214776: A method for automatically generating target program source code by inputting software specifications written in natural language Patent 7064680: A method for natural language processing added to JP Patent Publication 06-214776, and the corresponding program code stored in a database Patent 6958884: The content is similar to JP Patent Publication 06-21477, but adds a conversation reception function. Patent application 2007-175873: By giving a test scenario, efficient testing based on that test scenario can be performed Patent application 2023-145979: Verification method based on equivalence when combining algorithms at a higher level Patent application 2022-190529: A method for taking data flow as input, creating a model based on that, and deriving source code from this model Patent application 2021-189515: Automatic conversion of computer modeling environments to different models Patent application 2018-124226: A learning model evaluation method in which a second execution result based on a second learning model is obtained from an execution result based on a learning model that is a model using verification data, and the first and second execution results are judged to satisfy a logical formula and compared in behavior

In systems with automatic programming functions, the code generated by the automatic programming function for input text often differs from what was intended by the input text. For this reason, when using an automatic programming function, it is necessary for a human to execute the generated results, check them, and change the input to bring them closer to the originally intended program. This makes it possible to generate the intended program without human intervention.

A pattern that can exist as an input derived from the input text and a pattern that can exist as an output for that input are created. The input/output pattern is given to the program input and the output obtained is checked for equivalence. The program's accuracy is achieved by deactivating branches that cannot be checked for equivalence.

This prevents the automatic programming function from generating programs that include actions different from the user's intentions, eliminating the need for human intervention. In addition, programs are automatically created based on the input text without the human having to think about it, and by executing them, the generating AI can go beyond creating data to freely generate actions.

A process flow for masking a branch when a program's branching condition is not met in a system with AI generation or automatic programming capabilities Detailed flow of the flow in Figure 1 Details of determining the input format from the part of the input sentence corresponding to the input data in Figure 2 (#) Details of determining the output format from the part of the input sentence in Figure 2 that corresponds to the output data (#2) Create condition match flags for branches within the program, identify branches where the conditions do not match for the input data, and mask the conditions where the match flag is not set. After checking all input data, it is optional to leave the condition match flag or delete it.

The following describes in detail an embodiment of the present invention with reference to the drawings. However, the present invention can be implemented in many different forms, and should not be interpreted as being limited to the description of this embodiment. Note that the same elements are designated by the same numbers throughout the embodiments.

In the following embodiments, a method or system will be mainly described, but as will be apparent to those skilled in the art, the present invention can also be implemented as a program usable by a computer. Thus, the present invention can take the form of a hardware embodiment, a software embodiment, or an embodiment that combines software and hardware. The program can be recorded on any computer-readable medium, such as a hard disk, a DVD-ROM, an optical storage device, a magnetic storage device, or a semiconductor storage device.

In addition, in the following embodiments, a general computer system or the like can be used. The computer system, network system, and system for artificial intelligence that can be used in the embodiments include hardware resources that are generally provided in computer systems, network systems, and systems for artificial intelligence, such as a central processing unit (CPU), a main memory (RAM), a non-volatile memory (ROM), a coprocessor, a graphics processor (GPU), a cache memory, an input/output control device (I/O), semiconductors for artificial intelligence functions, and a flash memory. In addition, the system can include an external memory device such as a hard disk device or a flash memory, and a communication means that can be connected to a network such as the Internet. Computer systems include various computers such as personal computers, workstations, mainframe computers, and servers, cloud-based network-connected computers, tablet computers, various portable information terminals such as smartphones, and systems that constitute artificial intelligence.

The present invention aims to reduce the possibility that a generation AI or automatic program generation function, in so-called automatic programming, will output a program that differs from the program expected by the user, and to enable the automatic generation of a program that matches the text entered.

Automatic programming systems or generative AI with automatic programming functions are now able to automatically generate programs, that is, generate code, for input text using the automatic programming function. As generation accuracy improves, programs can be automatically generated and executed, and results can be obtained, even in situations where the user is not aware of it. This means that processing can be carried out without using application software. On the other hand, this could lead to new imitation and security risks.

Currently, programs (hereinafter referred to as programs or codes or program codes) generated using automatic programming functions often differ from the original intent of the input text.

One of the reasons different codes are generated is that a lack of learning leads to insufficient interpretation of the text, resulting in the output being different from what the program expects. This problem can be solved by deepening the AI's learning, and the accuracy can be improved, but it is known that accuracy cannot be improved beyond a certain level by learning context and other factors.

There are three possible reasons why accuracy does not reach a certain level. The first reason is a problem with the accuracy of the automatic creation of rules. AI uses either an arbitrary rule engine, or a rule engine itself that has been constructed by learning from accumulated information. In the latter case, the accuracy of the rule engine improves as the AI deepens its learning. However, since the accuracy changes as the information changes, it is known that it is not possible to improve beyond a certain level of accuracy. In other words, it is known that when accuracy reaches its limit, multiple options will continue to exist rather than a single unambiguous option. This is a cause that stems from the rule engine.

The second cause is that the input text can be interpreted differently depending on how it is interpreted. If there are multiple causes, the possibility of generating different codes increases accordingly. As a countermeasure, some methods involve a human combining the codes on a screen, etc. It is known that this problem can be solved to a certain extent by learning based on the context of the text, but it is known that accuracy beyond a certain level cannot be uniquely defined based on the context of the text. If the program created from the input text cannot be uniquely determined, the results will diverge due to the inability to define it, leading to the creation of an output that is different from the expected output. This is the second cause. The same thing can also happen when the input text does not contain enough information to make a judgment. In this case, the lack of information can only be resolved by inputting information equivalent to that of a human programming, but since the problem is caused by an inability to make a judgment, it cannot currently be solved by automatic programming.

The third cause occurs when processing data, and the input, or data pattern, differs depending on the source. For example, when generating a program to process information from the Internet, if the acquired data is used as input data, depending on the form of the data, it may contain data that does not match the program.

When generative AI or automatic programming becomes commonplace, results can be obtained by simply inputting input text that matches the purpose each time, without the need for a general-purpose application program. All that is required is to specify what input is used, what processing is performed, and how the results are to be output. Based on this premise, this invention solves the three causes mentioned above, thereby achieving accuracy beyond the limit of what can be achieved through learning, and making it possible to output the intended program. This is intended to create an environment in which users do not have to be conscious of the program, as the use environment for automatic programming becomes commonplace, and users can simply input what they want to do each time, and a program dedicated to the purpose is created and executed without them having to be conscious of it, resulting in the achievement of results. The premise is that the purpose is to solve a certain level of problems that cannot be solved by deep learning.

The solution to this problem is to limit the content of the sentences and phrases entered when automatically generating a program from a combination of sentences and phrases to the purpose, and to limit the input and output of the program to those specified or expected from the sentences and phrases entered, in other words, to a program that is limited to the content entered.

In automatic program generation, assuming the use of learning results, etc., the basis of this invention is to determine the input data and its format for the program to be generated before the program is generated, that is, to acquire the data and generate the program after reflecting the acquired results. The program is automatically generated as one that can accept the input data. Generally, when inputting data after a program is generated, if uncertain or unspecified data is acquired, there is a possibility that the program will cause unexpected behavior. To avoid this, it is necessary to identify and acquire what data can be input from the combination of sentences or phrases entered into the generation AI or automatic programming, and confirm the input. As shown in Figure 3, it is determined from what is specified in the input text, data acquired from external data, or from retained internal data.

For the program to be generated, input data is obtained from outside the system or from within the system, either from the input sentence or combination of phrases, or from the input sentence or combination of phrases. For example, if the target is within the system, data specified by the sentence or combination of phrases from the database is the target. Data obtained from the Internet as outside the system is taken as input data. The data to be output from that input is created from the input sentence or combination of phrases, or from the input data mentioned above. The output obtained by providing that input data to the program is compared with the output data mentioned above to perform an equivalence check, and the branch conditions in the program used for the matching combination are set as the branch conditions to be used. After performing an equivalence check for all inputs and outputs, unused branch conditions, that is, unused branch conditions, are masked or deleted. This results in a program that corresponds only to the input to be used. By masking unused internal branches (if and case) for the program to be generated using input and output data, it is possible to prevent program operations other than those in the program that can be executed using the input and output data mentioned above from occurring among the branches in the generated program.

The input and output to the program are extracted from the combination of input sentences and phrases based on what is obtained after the program is executed as a result of inputting data from the input document. The derived input uses the contents and format of the input data from the input document if they are available. If there is no format, the format is obtained from the data. If there is no data, data is obtained from outside the system and the data and its format are used. If data within the system is used, internal data is obtained and the data and its format are used. The output data and format are obtained from the combination of input sentences or phrases. If data is not included, a format uniquely determined from the sentence is used. If that is not used, a format obtained from the learned rules is used. These input data and formats are used to input the program, and the obtained output is compared with the input/output results of the program generated using the aforementioned output data and format. The branching conditions used by the data that matches are continued to be used, and all input/output data and formats are removed from the branching conditions that were not used. In other words, inactive branching conditions are excluded from the program's options, that is, from the options for branching conditions such as if and case. Finally, all unnecessary branching conditions are eliminated, and the program is generated again. In other words, the program has only branching conditions related to the input/output data. This does not make the program versatile, but makes it possible to prevent the generation of unintended programs by making the program dependent on input and output data. If a function is then added, the same processing as described above is carried out on the program generated from the combination of input sentences or phrases related to the function to be added, and then each function can be made into a separate program, or the common parts of each program can be combined into one program by performing function compression.

Figure 1 shows a flow chart outlining the solution. Extract the input/output data portion from the input sentences or combinations of phrases. Next, obtain the data to be input into the program from the input/output data portion. Next, automatically create a program from the input sentences or combinations of phrases. Next, input the input data into the program and obtain the output value. Compare the program output value with the output data from the input sentences or combinations of phrases. For combinations that match, record the branches in the program that were executed. Based on the results of executing all input/output data, mask all branches in the program that do not match the recorded branches, i.e., branches that were not passed through. In other words, modify the program to delete the branching portions of unused branches.

Figure 2 is a detailed flow diagram of Figure 1. From the combination of input sentences or phrases, a portion corresponding to input data and a portion corresponding to output data are extracted. A program is automatically created from the combination of input sentences or phrases. An input data format is determined from the portion of the combination of input sentences or phrases that corresponds to the input data to the program. Similarly, an output data format is determined from the portion corresponding to the output data. Next, data that matches the input data and input format is obtained or created. This increases the amount of input data. Similarly, data similar to the input data is derived from the combination of input sentences or phrases. As for the output data, output data that matches the output data and output format and corresponds to the increased input data is created. Next, the input data is input to the program and an output value is obtained. The program output value is compared with the output data from the combination of input sentences or phrases. For the combinations that match, the branch in the program that was executed is recorded. Based on the results of the comparison of all input and output data, all branches in the program that do not match the recorded branches, that is, unused branch conditions, are deleted. After the comparison using the final combination of input data and output data is completed, a program is generated from which unused branch conditions have been deleted.

Figure 3 is a flow diagram for determining the input format when the input data in Figure 2 is included in the input sentence or combination of phrases, when it has been obtained from the Internet or an external storage device, or when it is obtained from a database within the system, that is, a flow diagram for the part indicated by #. If the input sentence or combination of phrases includes the input data format, that format is used. If the input sentence or combination of phrases includes the input data or format, it is used, and if it does not include the format, the format is obtained from the data. If it does not include the format and the input data is external data, the external data is obtained from the storage source, sample data is obtained, and that format is used. If it does not include the format and the input data is internal data, the internal data is obtained from the storage source, sample data is obtained, and that format is used.

Figure 4 is a detailed flow diagram of the part shown in #2 in Figure 2, where the output format is included in the combination of input sentences or phrases, or is uniquely determined, or is determined from learned data or learned rules.

Figure 5 shows an example in which, when a condition is met in a branch within the program in Figure 2 when input data is entered, a condition match flag is set, and the branch conditions corresponding to condition match flags where the condition is not met, i.e., the condition match flag is not active, are masked. In the automatic programming of Figure 2, a condition match flag is created for each condition in all conditional branches, i.e., conditional branches such as If and Case. If the condition is met by the input data, this condition match flag is made active. After the final input data is entered, conditional branches with inactive condition match flags are masked. After that, it is up to the user to delete or leave all condition match flags.

Instead of using artificial intelligence systems based on past trends or arbitrary rules, this can be applied to systems with automatic programming capabilities that can generate rules based on the information the user expects. This will enable systems that can process information according to rules that meet the user's needs, such as systems where the user can automatically collect information from an unspecified number of sources, such as the Internet, process it, and specify the output format, allowing the user to obtain the processed information in the format they want each time.

This can be applied to systems with automatic programming functions that generate programs from acquired data. By using this, it is possible to create autonomous systems that can build execution methods based on the results of learning, such as robots that think and act on their own, and actually execute based on those execution methods.

Claims (1)

A generation AI system that generates a program based on a combination of sentences or phrases in the program content, has a function of acquiring input data to the program before generating the program, and generating only internal branching conditions related to the input data, and has an automatic program generation function that synthesizes the common parts of the programs generated when there are multiple combinations of sentences or phrases in the program content into one.

Images