CN114282686B

CN114282686B - Method and system for constructing a machine learning modeling process

Info

Publication number: CN114282686B
Application number: CN202111616468.XA
Authority: CN
Inventors: 徐昀; 娄辰; 张舒羽
Original assignee: 4Paradigm Beijing Technology Co Ltd
Current assignee: 4Paradigm Beijing Technology Co Ltd
Priority date: 2018-06-26
Filing date: 2018-06-26
Publication date: 2025-08-29
Anticipated expiration: 2038-06-26
Also published as: CN114282686A; CN108898229A; CN108898229B

Abstract

A method and system for constructing a machine learning modeling process are provided. The method includes: displaying the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process; executing at least one step in the machine learning modeling process in response to a user operation for executing the at least one step; while executing the at least one step, receiving a user modification operation for modifying the machine learning modeling process; and modifying the machine learning modeling process in response to the modification operation, wherein, when executing the modified portion of the machine learning modeling process, executing the process based on the modified machine learning modeling process. According to the method and system, the efficiency of machine learning modeling can be improved.

Description

Method and system for constructing machine learning modeling process

The application relates to a patent application with a filing date of 2018, 6, 26, 201810669281.8 and a patent application entitled "method and System for constructing a machine learning modeling Process".

Technical Field

The present invention relates generally to the field of machine learning, and more particularly, to a method and system for building a machine learning modeling process.

Background

With the advent of massive amounts of data, people have tended to use machine learning techniques to mine bid values from data. Machine learning is an inevitable consequence of the development of artificial intelligence research into a certain phase, which aims at improving the performance of the system itself by means of computation, using experience. In computer systems, "experience" is usually present in the form of "data" from which "models" can be generated by means of machine learning algorithms, i.e. by providing experience data to the machine learning algorithm, a model can be generated based on these experience data, which model provides corresponding decisions, i.e. predictions, in the face of new situations. It can be seen how to generate a model based on empirical data (i.e., a machine learning modeling process) is key to machine learning techniques.

When a user builds a machine learning modeling process, the machine learning modeling process needs to be continuously modified and operated until the built machine learning modeling process is confirmed to meet the requirements. At present, when a machine learning modeling process is built through a machine learning platform (for example, microsoft cloud computing system "Azure", etc.), in the process of running the built machine learning modeling process, only the configuration of steps in the machine learning modeling process can be checked, if a user wants to modify the machine learning modeling process, the whole machine learning modeling process can only be run out or the machine learning modeling process can be stopped manually, on one hand, waiting for the whole machine learning modeling process to run out will take a long time, which makes the machine learning modeling process unable to be completed timely, greatly influences the modeling efficiency, and on the other hand, the operation of the machine learning modeling process can be stopped manually, which may result in that the steps which have run for a long time are forced to stop running, so that part of the running results obtained by the steps are discarded, the running work is wasted, and the modeling efficiency is further influenced.

Disclosure of Invention

An exemplary embodiment of the present invention is to provide a method and a system for constructing a machine learning modeling process, so as to solve the problem of low modeling efficiency in the prior art.

According to an exemplary embodiment of the present invention, there is provided a method for constructing a machine learning modeling process, including displaying the constructed machine learning modeling process in a graphical interface for constructing a machine learning modeling process, executing at least one step of the machine learning modeling process in response to a user operation for executing the at least one step, receiving a modification operation for modifying the machine learning modeling process by a user while executing the at least one step, and modifying the machine learning modeling process in response to the modification operation, wherein when a modified portion of the machine learning modeling process is executed, the machine learning modeling process is executed based on the modified machine learning modeling process.

Optionally, the step of displaying the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process includes displaying a directed acyclic graph for representing the constructed machine learning modeling process in the graphical interface for constructing the machine learning modeling process, wherein nodes in the directed acyclic graph correspond one-to-one to steps in the machine learning modeling process.

Optionally, the modifying operation comprises a configuration operation for modifying a configuration of a step downstream of the step currently being executed, wherein the step of modifying the machine learning modeling process in response to the modifying operation comprises modifying a configuration of the step for which the configuration operation is directed while the at least one step is being executed in response to the configuration operation, wherein when a modified step is executed, the modified step is executed based on the modified configuration.

Optionally, the modifying operation comprises a configuration operation for modifying a configuration of a step currently being run or a configuration of a step upstream of the step currently being run, wherein the step of modifying the machine learning modeling process in response to the modifying operation comprises stopping running the step currently being run and modifying the configuration of the step for which the configuration operation is directed in response to the configuration operation, wherein the method further comprises starting running from the most upstream modified step when receiving again a user operation for running at least one step in the machine learning modeling process, wherein the modified step is run based on the modified configuration when running to the modified step.

Optionally, the modifying operation includes a structure adjustment operation for adjusting a structure downstream of a step currently being executed in the machine learning modeling process, wherein the step of modifying the machine learning modeling process in response to the modifying operation includes adjusting a structure of the machine learning modeling process while the at least one step is executed in response to the structure adjustment operation, wherein when executed to a portion of the machine learning modeling process that is adjusted structure, the machine learning modeling process is executed in accordance with the adjusted structure.

Optionally, the modifying operation comprises a structure adjustment operation for adjusting a structure upstream of a step currently being executed in the machine learning modeling process, wherein the step of modifying the machine learning modeling process in response to the modifying operation comprises stopping the step currently being executed in response to the structure adjustment operation and adjusting the structure of the machine learning modeling process, wherein the method further comprises starting the execution from a part of the structure adjusted upstream most of the machine learning modeling process when a user operation for executing at least one step in the machine learning modeling process is received again, wherein the execution is executed in accordance with the structure adjusted when the part of the structure adjusted to the machine learning modeling process is executed.

Optionally, the structure adjustment operation includes at least one of an operation for adding a step to the machine learning modeling process, an operation for deleting a step in the machine learning modeling process, and an operation for changing a logical relationship between steps in the machine learning modeling process.

Optionally, the method further comprises receiving a construction operation by a user to construct another machine learning modeling process in the graphical interface while the at least one step is being performed, and constructing the another machine learning modeling process while the at least one step is being performed in response to the construction operation.

Optionally, the step of receiving a user's modification operation for modifying the machine learning modeling process while running the at least one step includes receiving a user's selection operation of a node in the directed acyclic graph while running the at least one step, displaying a control for configuring a configuration item of a step corresponding to the selected node to the user in response to the selection operation, and receiving a user's input operation of the displayed control.

Optionally, the method further comprises the steps of receiving a selection operation of a node in the directed acyclic graph by a user while running the at least one step, displaying at least one control for displaying at least one output element of the step corresponding to the node around the selected node respectively in response to the selection operation, and displaying an output result of the output element corresponding to the selected control to the user in response to the selection operation of one of the at least one control by the user.

Optionally, the step of receiving a user's modification operation for modifying the machine learning modeling process while the at least one step is being performed includes displaying a list of nodes in a predetermined area of the graphical interface while the at least one step is being performed and receiving a user operation selecting and dragging a node from the list of nodes to connect to a node in the directed acyclic graph, and/or recommending to a user a node and/or a combination of nodes to which the node can connect through the connection point in response to a user operation for one connection point of the node in the directed acyclic graph while the at least one step is being performed, and receiving a user operation selecting a node or a combination of nodes from the recommended nodes and/or combinations of nodes to connect to the connection point.

Optionally, the method further comprises, in response to a user operation for running at least one step of the further machine learning modeling process, running the at least one step of the further machine learning modeling process while running the at least one step of the further machine learning modeling process, receiving a user modification operation for modifying the further machine learning modeling process while running the at least one step of the further machine learning modeling process, and modifying the further machine learning modeling process in response to the modification operation, wherein when running the modified portion of the further machine learning modeling process, running based on the modified machine learning modeling process.

According to another exemplary embodiment of the present invention, there is provided a system for constructing a machine learning modeling process, including a display device for displaying the constructed machine learning modeling process in a graphical interface for constructing a machine learning modeling process, an operation device for operating at least one step in the machine learning modeling process in response to a user operation for operating the at least one step, and a construction device for receiving a user modification operation for modifying the machine learning modeling process while the operation device is operating the at least one step and modifying the machine learning modeling process in response to the modification operation, wherein when the operation device is operating to the modified portion of the machine learning modeling process, the operation is based on the modified machine learning modeling process.

Optionally, the display device displays a directed acyclic graph for representing the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process, wherein nodes in the directed acyclic graph are in one-to-one correspondence with steps in the machine learning modeling process.

Optionally, the modifying operation includes a configuration operation for modifying a configuration of a step downstream of the step currently being operated, wherein, in response to the configuration operation, while the operating means operates the at least one step, the constructing means modifies the configuration of the step for which the configuration operation is directed, wherein, when the operating means operates to the modified step, the modified step is operated based on the modified configuration.

Optionally, the modifying operation includes a configuration operation for modifying a configuration of the step currently being operated or a configuration of a step upstream of the step currently being operated, wherein in response to the configuration operation, the operating means stops operating the step currently being operated, and the constructing means modifies the configuration of the step for which the configuration operation is directed, wherein when a user operation for operating at least one step in the machine learning modeling process is received again, the operating means starts operating from the most upstream modified step, wherein when the operating means operates to the modified step, the modified step is operated based on the modified configuration.

Optionally, the modifying operation includes a structure adjusting operation for adjusting a structure downstream of a step currently being executed in the machine learning modeling process, wherein in response to the structure adjusting operation, the constructing means adjusts the structure of the machine learning modeling process while the executing means executes the at least one step, wherein when the executing means is executed to a portion of the structure to which the machine learning modeling process is adjusted, the executing means is executed in accordance with the adjusted structure.

Optionally, the modifying operation includes a structure adjusting operation for adjusting a structure upstream of a step currently being executed in the machine learning modeling process, wherein in response to the structure adjusting operation, the executing means stops executing the step currently being executed, and the constructing means adjusts the structure of the machine learning modeling process, wherein when a user operation for executing at least one step in the machine learning modeling process is received again, the executing means starts executing from a part of the structure adjusted upstream of the machine learning modeling process, wherein when the executing means executes to the part of the structure adjusted by the machine learning modeling process, the executing means executes in accordance with the structure adjusted.

Optionally, the constructing means receives a construction operation by a user to construct another machine learning modeling process in the graphical interface while the operating means operates the at least one step, and constructs the another machine learning modeling process while the operating means operates the at least one step in response to the construction operation.

Optionally, the constructing device receives a selection operation of the user on the nodes in the directed acyclic graph while the operating device operates the at least one step, displays a control for configuring a configuration item of the step corresponding to the selected node to the user in response to the selection operation, and receives an input operation of the user on the displayed control.

Optionally, the construction device receives the selection operation of the user on the nodes in the directed acyclic graph while the operation device operates the at least one step, displays at least one control for displaying at least one output element of the step corresponding to the node around the selected nodes in response to the selection operation, and displays the output result of the output element corresponding to the selected control to the user in response to the selection operation of one of the at least one control by the user.

Optionally, the construction means displays a list of nodes in a predetermined area of the graphical interface and receives a user operation to select and drag a node from the list of nodes to connect to a node in the directed acyclic graph while the operation means is running the at least one step, and/or, the construction means recommends to the user a node and/or a combination of nodes to which the node can connect through the connection point in response to a user operation to one connection point of the node in the directed acyclic graph while the operation means is running the at least one step, and receives a user operation to select a node or a combination of nodes from the recommended nodes and/or combinations of nodes to connect to the connection point.

Optionally, the means for operating is responsive to user operation for operating at least one step of the further machine learning modeling process, while operating the at least one step of the further machine learning modeling process, means for receiving a user modification operation for modifying the further machine learning modeling process while the means for operating is operating the at least one step of the further machine learning modeling process, and responsive to the modification operation, modifying the further machine learning modeling process, wherein the means for operating is operated based on the modified machine learning modeling process when the means for operating is operating to the modified portion of the further machine learning modeling process.

According to another exemplary embodiment of the present invention, a computer readable medium is provided, on which a computer program for executing the method for constructing a machine learning modeling process as described above is recorded.

According to another exemplary embodiment of the present invention, a computing device is provided, comprising a storage means and a processor, wherein the storage means has stored therein a set of computer executable instructions which, when executed by the processor, perform a method for constructing a machine learning modeling process as described above.

According to the method and the system for constructing the machine learning modeling process of the exemplary embodiment of the invention, the machine learning modeling process can be modified while the machine learning modeling process is running. In addition, one machine learning modeling process can be run while another machine learning modeling process is being constructed and/or run.

According to the method and the system for constructing the machine learning modeling process, the operation mode of the machine learning modeling process is more flexible, on one hand, a user can conveniently continue to perfect the machine learning modeling process (for example, modifying structure or step configuration, completing other steps of the machine learning modeling process and the like) during the operation of one machine learning modeling process, or construct another machine learning modeling process, so that the user can utilize the operation time of the machine learning modeling process to perform other modeling work, namely, the efficiency of the modeling work is improved by improving the parallelism of the operation work, and on the other hand, the user can conveniently adjust the configuration or the downstream structure of the downstream step in time based on the operation effect (for example, output result) of the upstream step, so that the downstream part can better perform suitable processing on the operation result of the upstream step, and the flexibility and the efficiency of the modeling work are greatly improved.

Additional aspects and/or advantages of the present general inventive concept will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the general inventive concept.

Drawings

The foregoing and other objects and features of exemplary embodiments of the invention will become more apparent from the following description taken in conjunction with the accompanying drawings which illustrate exemplary embodiments in which:

FIG. 1 illustrates a flowchart of a method for constructing a machine learning modeling process according to an exemplary embodiment of the present invention;

FIG. 2 illustrates an example of a build machine learning modeling process according to an exemplary embodiment of the present invention;

FIG. 3 illustrates an example of a control for presenting output elements of steps corresponding to a selected node according to an exemplary embodiment of the present invention;

FIG. 4 illustrates an example of presenting output results of steps corresponding to a selected node to a user according to an exemplary embodiment of the present invention;

FIG. 5 illustrates another example of a build machine learning modeling process according to an exemplary embodiment of the present invention;

FIG. 6 shows a block diagram of a system for building a machine learning modeling process, according to an example embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments will be described below in order to explain the present invention by referring to the figures.

FIG. 1 shows a flowchart of a method for constructing a machine learning modeling process, according to an exemplary embodiment of the invention. Here, the method may be performed by a computer program, or may be performed by a dedicated hardware device or an aggregate of software and hardware resources for performing machine learning, big data calculation, or data analysis, for example, a machine learning platform for implementing a machine learning-related service, as an example.

Referring to fig. 1, in step S10, a constructed machine learning modeling process is displayed in a graphical interface for constructing the machine learning modeling process.

As an example, a graphical interface for building a machine learning modeling process may be entered first, and then, when an operation of a user to open a file representing the machine learning modeling process is received, the machine learning modeling process defined by the content of the file may be displayed in the graphical interface, or when an operation of a user to create the machine learning modeling process is received, the built machine learning modeling process may be displayed in the graphical interface in real time in response to a building operation of the user for building the machine learning modeling process.

And, the user may continue building the displayed machine learning modeling process through the graphical interface before the built machine learning modeling process is not run. For example, the configuration of the step for which the configuration operation is directed may be modified in response to a configuration operation of a user for modifying the configuration of the step, and the structure of the machine learning modeling process may be adjusted in response to a structure adjustment operation of a user for adjusting the structure of the machine learning modeling process. As an example, the structure adjustment operation may include at least one of an operation for adding a step to the machine learning modeling process, an operation for deleting a step in the machine learning modeling process, and an operation for changing a logical relationship between steps in the machine learning modeling process. Here, the logical relationship between steps, that is, the input-output association relationship between steps, for example, the output of one step serves as the input of another step.

By way of example, the constructed machine learning modeling process may include at least one of data importation, data stitching, data splitting, feature extraction, model training, model testing, and model evaluation. Specifically, the data importing step is used for importing one or more data sets (e.g., data tables) containing historical data records, the data stitching step is used for stitching the data records in the imported data sets, the data splitting step is used for splitting the stitched data records into a training set and a testing set, or splitting the data records in the imported data set into the training set and the testing set, wherein the data records in the training set are used for being converted into training samples to train out models, the data records in the testing set are used for being converted into testing samples to evaluate model effects according to the test results of the trained models on the testing samples, the feature extracting step is used for extracting features of the training set and the testing set to generate training samples and testing samples, the model training step is used for training out a machine learning model based on the training samples according to a machine learning algorithm, the model testing step is used for obtaining the test results of the trained machine learning model on the testing samples, and the model evaluating step is used for evaluating the effects of the trained machine learning model based on the accuracy of the testing results.

As an example, a directed acyclic graph (DAG graph) for representing a constructed machine learning modeling process may be displayed in a graphical interface for constructing the machine learning modeling process, wherein nodes in the directed acyclic graph correspond one-to-one to steps in the machine learning modeling process. The user may construct or run a corresponding machine learning modeling process through editing operations or running operations on the displayed DAG graph.

In step S20, at least one step in the machine learning modeling process is executed in response to a user operation for executing the at least one step.

Here, as an example, a user selection of one or more executable steps in the machine learning modeling process may be received and the corresponding steps may be executed sequentially according to the user selection.

In step S30, while the at least one step is being performed, a modification operation of a user for modifying the machine learning modeling process is received.

By way of example, the modifying operation may include a configuration operation for modifying a configuration of steps in the machine learning modeling process and/or a structure adjustment operation for adjusting a structure of the machine learning modeling process. As an example, the structure adjustment operation may include at least one of an operation for adding a step to the machine learning modeling process, an operation for deleting a step in the machine learning modeling process, and an operation for changing a logical relationship between steps in the machine learning modeling process. As an example, when a directed acyclic graph for representing a built machine learning modeling process is displayed in a graphical interface for building the machine learning modeling process, the structure adjustment operation may include at least one of adding nodes to the directed acyclic graph, deleting nodes from the directed acyclic graph, and changing connection relationships between nodes in the directed acyclic graph.

In step S40, the machine learning modeling process is modified in response to the modifying operation, wherein when running to the modified portion of the machine learning modeling process, a modified machine learning modeling process is run based.

As an example, when the modifying operation is a configuration operation for modifying the configuration of a step downstream of a step currently being executed, the configuration of the step for which the configuration operation is directed is modified while the at least one step is being executed in response to the configuration operation, wherein when the modified step is executed, the modified step is executed based on the modified configuration.

Here, the step downstream of the step currently being operated means a step in which the operation order should follow the step currently being operated. It should be understood that the step targeted by the configuration operation may be a step that will be performed in the present operation (i.e., a step in the at least one step), or may be a step that will not be performed in the present operation (i.e., a step other than the step in the at least one step).

As an example, when the modifying operation is a structure adjusting operation for adjusting a structure downstream of a step currently being executed in the machine learning modeling process, the structure of the machine learning modeling process is adjusted while the at least one step is executed in response to the structure adjusting operation, wherein when the modifying operation is executed to a portion of the machine learning modeling process to which the structure is adjusted, the modifying operation is executed in accordance with the adjusted structure.

Here, the downstream structure of the step currently being executed in the machine learning modeling process, that is, the structure of the portion of the machine learning modeling process whose execution order should follow the step currently being executed. It should be understood that the structure adjustment operation may be used to adjust the structure of a portion that will be operated in the present operation, and may also be used to adjust the structure of a portion that will not be operated in the present operation.

According to the above-described exemplary embodiments of the present invention, the configuration or downstream structure of the downstream step of the step currently being operated can be flexibly adjusted without affecting the operation of the machine learning modeling process.

As an example, when the modifying operation is a configuration operation for modifying a configuration of a step currently being executed or a configuration of a step upstream of the step currently being executed, the step currently being executed is stopped and the configuration of the step for which the configuration operation is directed is modified in response to the configuration operation, wherein the method for constructing a machine learning modeling process according to an exemplary embodiment of the present invention further includes, when a user operation for executing at least one step in the machine learning modeling process is received again, starting from the most upstream modified step, wherein when executing to the modified step, executing the modified step based on the modified configuration.

Here, the step upstream of the step currently being operated means a step in which the operation order should precede the step currently being operated. It should be understood that the step targeted by the configuration operation may be a step that has been already performed or is being performed in the present operation (i.e., is a step in the at least one step), or may be a step that is not performed in the present operation (i.e., is not a step in the at least one step). Here, the most upstream modified step is the step whose execution order is the most front among the modified steps. Since the upstream step is modified, the output result thereof is necessarily changed accordingly to affect the operation of the downstream step, and therefore, when the user operation for operating at least one step in the machine learning modeling process is received again, even if the at least one step does not include the most upstream modified step, the operation needs to be started from the most upstream modified step.

As an example, when the modifying operation is a structure adjusting operation for adjusting an upstream structure of a step currently being executed in the machine learning modeling process, the step currently being executed is stopped to be executed in response to the structure adjusting operation, and the structure of the machine learning modeling process is adjusted, wherein the method for constructing the machine learning modeling process according to the exemplary embodiment of the present invention further includes starting to be executed from a part of the structure being adjusted upstream of the machine learning modeling process when a user operation for executing at least one step in the machine learning modeling process is received again, wherein the operation is executed in accordance with the structure after being adjusted when the part of the structure being adjusted to the machine learning modeling process is executed.

Here, the upstream structure of the step currently being executed in the machine learning modeling process, that is, the structure of the portion of the machine learning modeling process whose execution order should precede the step currently being executed. It should be understood that the structure adjustment operation may be used to adjust the structure of a portion that has been operated in the present operation, and may also be used to adjust the structure of a portion that has not been operated in the present operation.

As an example, steps S20 to S40 may be implemented by the method being executable by a machine learning platform for executing a machine learning process, and in response to a user operation for executing at least one step in a displayed machine learning modeling process, a task for executing the corresponding step may be submitted to an execution device of the machine learning platform (e.g., a server for executing a task workflow at a rear end of the machine learning platform) to sequentially execute the at least one step in accordance with a configuration of the at least one step and a logical relationship between the steps. While the running means is running the at least one step, receiving a configuration operation, and in response to the configuration operation, determining which type the step for which is the step to be run downstream of the step currently being run (i.e. the step in the running queue), the step not to be run downstream of the step currently being run (i.e. the step not in the running queue), the step currently being run and the steps upstream thereof. When determining a step to be operated downstream of a step currently being operated, modifying the configuration of the step aimed at by the configuration operation, and informing an operating device that the configuration of the step has been modified, so that the operating device operates the step according to the modified configuration when operating the step, when determining a step not to be operated downstream of the step currently being operated, modifying the configuration of the step aimed at by the configuration operation, when determining a step belonging to the step currently being operated and an upstream step thereof, informing the operating device to stop operating the at least one step, and modifying the configuration of the step aimed at by the configuration operation.

Accordingly, while the operation device is operating the at least one step, a structure adjustment operation is received, and in response to the structure adjustment operation, it is determined which type the part for which the structure adjustment operation is directed belongs to, namely a downstream structure of the currently operating step including the step to be operated downstream of the currently operating step, a downstream structure of the currently operating step excluding the step to be operated downstream of the currently operating step, and an upstream structure of the currently operating step. When determining a downstream structure of a currently running step which belongs to a step to be run including a step downstream of the currently running step, the structure of a part for which the structure adjustment operation is directed is adjusted, and the operating device is notified that the structure of the part has been modified so that the operating device operates in accordance with the modified structure when running the part, when determining a downstream structure of a step to be run which does not include a step downstream of the currently running step, the structure of the part for which the structure adjustment operation is directed is adjusted, and when determining an upstream structure of a step which belongs to the currently running step, the operating device is notified to stop running the at least one step, and the structure of the part for which the structure adjustment operation is directed is adjusted.

As an example, when a directed acyclic graph representing a built machine learning modeling process is displayed in a graphical interface for building the machine learning modeling process and a step corresponding to at least one node in the directed acyclic graph is executed, a configuration operation may be received while the at least one step is executed by receiving a user selection operation of a node in the directed acyclic graph while the at least one step is executed, displaying a control for configuring a configuration item of the step corresponding to the selected node to the user in response to the selection operation, and receiving an input operation of the displayed control by the user. For example, the selection operation of the node may be an operation of clicking the node by a left mouse button. For example, a control for configuring a configuration item of a step corresponding to the selected node may be displayed in a predetermined area of the graphical interface.

As an example, when a directed acyclic graph representing a machine learning modeling process is displayed in a graphical interface for building the machine learning modeling process and a step corresponding to at least one node in the directed acyclic graph is executed, a structure adjustment operation may be received while the at least one step is executed by displaying a list of nodes in a predetermined area of the graphical interface and receiving a user operation of selecting and dragging a node from the list of nodes to connect to a node in the directed acyclic graph while the at least one step is executed, and/or, in response to a user operation of a connection point for a node in the directed acyclic graph while the at least one step is executed, recommending a node and/or a combination of nodes to which the node is connectable through the connection point to a user and receiving an operation of the user selecting the node or the combination of nodes from the recommended node and/or the combination of nodes to connect to the connection point. For example, nodes and/or node combinations to which the node is connectable through the connection point may be shown around the connection point. For example, the user operation for one connection point of one node in the directed acyclic graph may include hovering over one connection point of one node in the directed acyclic graph and clicking on the connection point after the connection point enters a to-be-connected state in response to the hovering operation.

As shown in fig. 2, a DAG graph representing a machine learning modeling process is displayed in a graphical interface for constructing the machine learning modeling process, a step corresponding to a node "HE-TreeNet" in the DAG graph is currently being operated, and while continuing to operate a step corresponding to a node "HE-TreeNet", a control for configuring a configuration item of a model test step may be displayed in a right region of the graphical interface in response to a user's selection operation of the model test node, and the configuration of the model test step is modified in response to a user's input operation of the displayed control, thereby implementing an adjustment of the configuration of the model test step to operate the model test step based on the adjusted configuration, before operating the model test step. In addition, while the step corresponding to the node "HE-TreeNet" is being operated, a control for configuring a configuration item of the feature extraction step may be displayed in the right area of the graphical interface in response to a user's selection operation of the feature extraction node, then, in response to a user's input operation of the displayed control, the step corresponding to the node "HE-TreeNet" is stopped and the configuration of the feature extraction step is modified, and when the user's operation for operating the DAG graph is received again, since none of the steps upstream of the feature extraction step is changed, the operation may be started from the feature extraction step and the feature extraction step is operated based on the modified configuration. In addition, a node list may be displayed in a left area of the graphical interface, and a structure adjustment operation for selecting and dragging a node from the node list to connect to a node in the directed acyclic graph may be received while running the corresponding step of "HE-TreeNet" nodes.

Further, as an example, the method for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may further include, when a directed acyclic graph for representing the constructed machine learning modeling process is displayed in a graphical interface for constructing a machine learning modeling process and a step corresponding to at least one node in the directed acyclic graph is executed, receiving a user selection operation of the node in the directed acyclic graph, and may display at least one control for displaying at least one output element respectively for displaying the step corresponding to the node around the selected node in response to the selection operation, and display an output result of the output element corresponding to the selected control to the user in response to the user selection operation of one of the at least one control. At least one output element of a step is at least one element output by the step. As an example, the at least one control may correspond one-to-one with the at least one output element. As an example, the at least one control may be applied with a corresponding visual effect by a type of the corresponding output element, wherein different types of corresponding visual effects are different. As an example, the output results of the output element may include a current output result and/or a historical output result of the output element. Here, the current output result is an output result obtained after the step corresponding to the selected node is performed this time, and the historical output result is an output result obtained after the step corresponding to the selected node is performed before the current operation. As an example, the visual effect of the at least one control being displayed may also be used to distinctively indicate whether the corresponding output element has the result of the current run.

As an example, the output result of the output element may be the specific output content itself of the output element, or may be information related to the specific output content, for example, may be the size of the specific output content, a channel entry for accessing the specific output content, or the like. It should be understood that the types of the plurality of output elements of the same step may be the same or different, and the types of the output elements of different steps may be the same or different. By way of example, the types of output elements may include at least one of a data table, information defining a machine learning model, an assessment report, an analysis report. For example, the data table may be a data table as a training set and a data table as a test set output by the data splitting step, a data table as a training sample and a data table as a test sample output by the feature extracting step, or a data table indicating a test result output by the model testing step, the information for defining the machine learning model may be a parameter of the machine learning model, the evaluation report may be a report for evaluating a test effect of the machine learning model, and the analysis report may be a report on analysis performed during the operation step, for example, a report on feature importance analysis performed during the operation of the feature extracting step.

As shown in fig. 3, in response to a user's selection operation of a node in the running directed acyclic graph, at least one control for respectively showing at least one output element of a step corresponding to the node may be displayed around the selected node, types of a plurality of output elements of the step may be the same or different, and controls corresponding to the different types of output elements are differently displayed. As shown in fig. 4, the output results of the output elements corresponding to the selected control may be presented in the right region of the graphical interface in response to a user's selection operation of the control displayed around the data splitting node, and the connection line between the data splitting node and the subsequent node (i.e., feature extraction node) to which the output element corresponding to the selected control is applied may be highlighted (e.g., highlighted). The output result of the data splitting step can be displayed to the user by displaying the size of the specific output content of the output element corresponding to the selected control, the channel entrance for accessing the specific output content, and the like, and in addition, the display of the current output result and the historical output result can be switched according to the selection of the user. It should be understood that the specific interaction scenario and operational details of the exemplary embodiments of the present invention in presenting the output results of the steps corresponding to the selected nodes to the user are not limited to the examples shown in fig. 3 and 4.

According to the above-mentioned exemplary embodiment of the present invention, it is convenient for a user to look up the output result of the upstream step, and to adjust the configuration or downstream structure of the downstream step in time, so that the downstream portion can better perform suitable processing on the output result of the upstream step, and the flexibility and efficiency of the modeling work are greatly improved.

Further, as an example, a method for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may further include receiving a construction operation by a user to construct another machine learning modeling process in the graphical interface while the at least one step is being executed, and constructing the other machine learning modeling process while the at least one step is being executed in response to the construction operation.

Further, as an example, a method for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may further include, in response to a user operation for executing at least one step in the other machine learning modeling process, executing the at least one step in the other machine learning modeling process while executing the at least one step in the machine learning modeling process. According to an exemplary embodiment of the present invention, a plurality of machine learning modeling processes can be simultaneously run, and the operations thereof do not affect each other.

Further, as an example, the method for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may further include receiving a modification operation of a user for modifying the other machine learning modeling process while the at least one step of the other machine learning modeling process is being executed, and modifying the other machine learning modeling process in response to the modification operation, wherein the modified machine learning modeling process is executed based on the modified machine learning modeling process when the modified portion of the other machine learning modeling process is executed.

As shown in fig. 5, a DAG graph representing a machine learning modeling process is displayed in a graphical interface for constructing the machine learning modeling process, a step corresponding to a node "HE-TreeNet" in the DAG graph is currently running, a node list may be displayed in a left region of the graphical interface, a canvas region selected from the node list and added to the graphical interface may be received while the DAG graph is continued to be run, and a user operation to connect newly added nodes to each other may be received, and another DAG graph may be created in response to the user operation. In addition, the newly created DAG graph may be run at the same time as the original DAG graph is run in response to a user operation. Further, a user's modification operation for modifying the newly created DAG graph may also be received while the newly created DAG graph is being run, and the newly created DAG graph is modified in response to the modification operation, wherein when the modified portion of the newly created DAG graph is run, the modified DAG graph is run based on the modified DAG graph. It should be appreciated that the specific interaction scenario and operational details of running the machine learning modeling process according to an exemplary embodiment of the present invention are not limited to the examples shown in fig. 2 and 5.

FIG. 6 shows a block diagram of a system for building a machine learning modeling process, according to an example embodiment of the present invention. As shown in fig. 6, a system for constructing a machine learning modeling process according to an exemplary embodiment of the present invention includes a display device 10, an operating device 20, and a constructing device 30.

Specifically, the display device 10 is used to display the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process.

As an example, the display device 10 may display a directed acyclic graph representing a constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process, wherein nodes in the directed acyclic graph correspond one-to-one to steps in the machine learning modeling process.

The running means 20 is for running at least one step in the machine learning modeling process in response to a user operation for running the at least one step.

The construction means 30 is configured to receive a modification operation by a user for modifying the machine learning modeling process while the execution means 20 is executing the at least one step, and to modify the machine learning modeling process in response to the modification operation, wherein when the execution means 20 is executing the modified part of the machine learning modeling process, the execution is based on the modified machine learning modeling process.

As an example, the modifying operation may comprise a configuration operation for modifying a configuration of a step downstream of the step currently being operated, wherein, in response to the configuration operation, while the operating device 20 is operating the at least one step, the build device 30 may modify the configuration of the step for which the configuration operation is directed, wherein, when the operating device 20 is operating to the modified step, the modified step is operated based on the modified configuration.

As an example, the modifying operation may include a configuration operation for modifying a configuration of a step currently being executed or a configuration of a step upstream of the step currently being executed, wherein in response to the configuration operation, the executing apparatus 20 stops executing the step currently being executed, and the constructing apparatus 30 modifies the configuration of the step for which the configuration operation is directed, wherein when a user operation for executing at least one step in the machine learning modeling process is received again, the executing apparatus 20 starts executing from the most upstream modified step, wherein when the executing apparatus 20 executes to the modified step, the modified step is executed based on the modified configuration.

As an example, the modifying operation may include a structure adjusting operation for adjusting a structure downstream of a step currently being executed in the machine learning modeling process, wherein, in response to the structure adjusting operation, the constructing means 30 adjusts the structure of the machine learning modeling process while the executing means 20 is executing the at least one step, wherein, when the executing means 20 is executed to a portion of the structure to which the machine learning modeling process is adjusted, the executing means is executed in accordance with the adjusted structure.

As an example, the modifying operation may include a structure adjusting operation for adjusting a structure upstream of a step currently being executed in the machine learning modeling process, wherein in response to the structure adjusting operation, the executing device 20 stops executing the step currently being executed, and the constructing device 30 adjusts the structure of the machine learning modeling process, wherein when a user operation for executing at least one step in the machine learning modeling process is received again, the executing device 20 starts executing from a part of the structure adjusted upstream of the machine learning modeling process, wherein when the executing device 20 executes to the part of the structure adjusted by the machine learning modeling process, it executes in accordance with the structure adjusted.

As an example, the structure adjustment operation may include at least one of an operation for adding a step to the machine learning modeling process, an operation for deleting a step in the machine learning modeling process, and an operation for changing a logical relationship between steps in the machine learning modeling process.

As an example, the constructing apparatus 30 may receive a user's selection operation of a node in the directed acyclic graph while the operating apparatus 20 operates the at least one step, display a control for configuring a configuration item of a step corresponding to the selected node to the user in response to the selection operation, and receive an input operation of the displayed control by the user.

As an example, the construction means 30 may receive a user's selection operation of a node in the directed acyclic graph while the operation means 20 operates the at least one step, display at least one control for presenting at least one output element of a step corresponding to the node, respectively, around the selected node in response to the selection operation, and present an output result of the output element corresponding to the selected control to the user in response to the user's selection operation of one of the at least one control.

As an example, the construction means 30 may display a node list at a predetermined area of the graphical interface while the operation means 20 operates the at least one step, and receive a user operation of selecting and dragging a node from the node list to connect to a node in the directed acyclic graph.

As an example, the construction means 30 may recommend to the user, in response to a user operation for one connection point of one node in the directed acyclic graph, a node and/or a combination of nodes to which the node can be connected through the connection point while the operation means 20 operates the at least one step, and receive an operation in which the user selects one node or a combination of nodes from the recommended nodes and/or combinations of nodes to connect to the connection point.

As an example, the construction device 30 may receive a construction operation by which a user constructs another machine learning modeling process in the graphical interface while the operation device 20 is operating the at least one step, and construct the another machine learning modeling process while the at least one step is operating in response to the construction operation.

As an example, the running means 20 may run at least one step in the other machine learning modeling process while running the at least one step in the machine learning modeling process in response to a user operation for running the at least one step in the other machine learning modeling process.

As an example, the constructing means 30 may receive a modification operation by a user for modifying the other machine learning modeling process while the operating means 20 is operating the at least one step in the other machine learning modeling process, and modify the other machine learning modeling process in response to the modification operation, wherein when the operating means 20 is operating the modified portion of the other machine learning modeling process, it is operated based on the modified machine learning modeling process.

It should be appreciated that the specific implementation of the system for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may be implemented with reference to the related specific implementations described in connection with fig. 1 to 5, and will not be described herein.

The apparatus included in the system for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may be configured as software, hardware, firmware, or any combination thereof, respectively, that performs a specific function. For example, these means may correspond to application specific integrated circuits, to pure software code, or to modules of software in combination with hardware. Furthermore, one or more functions implemented by these means may also be performed uniformly by components in a physical entity apparatus (e.g., a processor, a client, a server, or the like).

It should be appreciated that the method for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may be implemented by a program recorded on a computer readable medium, for example, according to an exemplary embodiment of the present invention, a computer readable medium for constructing a machine learning modeling process may be provided, wherein a computer program for executing steps of the method is recorded on the computer readable medium, wherein the constructed machine learning modeling process is displayed in a graphical interface for constructing the machine learning modeling process, the at least one step is executed in response to a user operation for executing the at least one step in the machine learning modeling process, a modification operation for modifying the machine learning modeling process is received by a user while the at least one step is executed, and the machine learning modeling process is modified in response to the modification operation, wherein the execution is based on the modified machine learning modeling process when the modified portion of the machine learning modeling process is executed.

The computer program in the above-described computer readable medium may be run in an environment deployed in a computer device such as a client, a host, a proxy device, a server, etc., and it should be noted that the computer program may also be used to perform additional steps other than the above-described steps or to perform more specific processes when the above-described steps are performed, and the contents of these additional steps and further processes have been described with reference to fig. 1 to 5, and will not be repeated here.

It should be noted that the system for constructing a machine learning modeling process according to an exemplary embodiment of the present invention may completely rely on the execution of a computer program to implement the corresponding functions, i.e., each device corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a specific software package (e.g., lib library) to implement the corresponding functions.

On the other hand, each of the devices included in the system for constructing the machine learning modeling process according to the exemplary embodiment of the present invention may also be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the corresponding operations may be stored in a computer-readable medium, such as a storage medium, so that the processor can perform the corresponding operations by reading and executing the corresponding program code or code segments.

For example, exemplary embodiments of the invention may also be implemented as a computing device comprising a storage component and a processor, the storage component having stored therein a set of computer-executable instructions that, when executed by the processor, perform a method for constructing a machine learning modeling process.

In particular, the computing devices may be deployed in servers or clients, as well as on node devices in a distributed network environment. Further, the computing device may be a PC computer, tablet device, personal digital assistant, smart phone, web application, or other device capable of executing the above-described set of instructions.

Here, the computing device need not be a single computing device, but may be any device or collection of circuits capable of executing the above-described instructions (or instruction set) alone or in combination. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces with locally or remotely (e.g., via wireless transmission).

In the computing device, the processor may include a Central Processing Unit (CPU), a Graphics Processor (GPU), a programmable logic device, a special purpose processor system, a microcontroller, or a microprocessor. By way of example, and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.

Some of the operations described in the method for constructing a machine learning modeling process according to the exemplary embodiment of the present invention may be implemented in software, some of the operations may be implemented in hardware, and furthermore, the operations may be implemented in a combination of software and hardware.

The processor may execute instructions or code stored in one of the storage components, wherein the storage component may also store data. Instructions and data may also be transmitted and received over a network via a network interface device, which may employ any known transmission protocol.

The memory component may be integrated with the processor, for example, RAM or flash memory disposed within an integrated circuit microprocessor or the like. Further, the storage component may comprise a stand-alone device, such as an external disk drive, a storage array, or any other storage device usable by a database system. The storage component and the processor may be operatively coupled or may communicate with each other, such as through an I/O port, network connection, etc., such that the processor is able to read files stored in the storage component.

In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be connected to each other via buses and/or networks.

Operations involved in a method for constructing a machine learning modeling process according to exemplary embodiments of the present invention may be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams may be equally integrated into a single logic device or operate at non-exact boundaries.

For example, as described above, a computing device for building a machine learning modeling process according to an exemplary embodiment of the present invention may include a storage unit and a processor, wherein the storage unit stores a set of computer-executable instructions that, when executed by the processor, perform the steps of displaying the built machine learning modeling process in a graphical interface for building the machine learning modeling process, executing at least one step of the machine learning modeling process in response to a user operation for executing the at least one step, receiving a modification operation by a user for modifying the machine learning modeling process while executing the at least one step, and modifying the machine learning modeling process in response to the modification operation, wherein the execution is based on the modified machine learning modeling process when executing the modified portion of the machine learning modeling process.

The foregoing description of exemplary embodiments of the invention has been presented only to be understood as illustrative and not exhaustive, and the invention is not limited to the exemplary embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. Therefore, the protection scope of the present invention shall be subject to the scope of the claims.

Claims

1. A method for constructing a machine learning modeling process, comprising:

Displaying the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process;

In response to a user operation for executing at least one step in the machine learning modeling process, executing the at least one step, wherein the at least one step includes at least one of data importation, data stitching, data splitting, feature extraction, model training, model testing, and model evaluation;

Receiving a modification operation of a user for modifying the machine learning modeling process while the at least one step is running;

In response to the modifying operation, modifying the machine learning modeling process,

Wherein when running to the modified portion of the machine learning modeling process, running based on the modified machine learning modeling process;

Wherein the modifying operation includes a configuration operation for modifying a configuration of a step currently being run or a configuration of a step upstream of the step currently being run, wherein the step of modifying the machine learning modeling process in response to the modifying operation includes:

in response to the configuration operation, stopping the currently running step, and modifying the configuration of the step for which the configuration operation is directed,

Wherein the method further comprises, when a user operation for running at least one step in the machine learning modeling process is received again, starting from the modified step furthest upstream,

Wherein when running to a modified step, the modified step is run based on the modified configuration.

2. The method of claim 1, wherein displaying the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process comprises:

a directed acyclic graph representing a constructed machine learning modeling process is displayed in a graphical interface for constructing the machine learning modeling process, wherein nodes in the directed acyclic graph are in one-to-one correspondence with steps in the machine learning modeling process.

3. The method of claim 1, wherein the modifying operation comprises a configuration operation for modifying a configuration of a step downstream of a step currently running, wherein modifying the machine learning modeling process in response to the modifying operation comprises:

in response to the configuration operation, modifying the configuration of the step for which the configuration operation is directed while the at least one step is being run,

4. The method of claim 1, wherein the modifying operation comprises a structure adjustment operation for adjusting a structure downstream of a step currently running in the machine learning modeling process, wherein modifying the machine learning modeling process in response to the modifying operation comprises:

Adjusting a structure of the machine learning modeling process while the at least one step is being performed in response to the structure adjustment operation,

Wherein when running to a portion of the machine learning modeling process adjusted structure, the machine learning modeling process is run according to the adjusted structure.

5. The method of claim 1, wherein the modifying operation comprises a structure adjustment operation for adjusting a structure upstream of a step currently running in the machine learning modeling process, wherein modifying the machine learning modeling process in response to the modifying operation comprises:

Stopping the currently running step in response to the structure adjustment operation, and adjusting the structure of the machine learning modeling process,

Wherein the method further comprises, when a user operation for running at least one step in the machine learning modeling process is received again, starting running from a part of the structure that is adjusted upstream most of the machine learning modeling process,

6. The method of claim 4 or 5, wherein the structure adjustment operation comprises at least one of an operation for adding steps to a machine learning modeling process, an operation for deleting steps in a machine learning modeling process, and an operation for changing logical relationships between steps in a machine learning modeling process.

7. The method of claim 1, wherein the method further comprises:

Receiving a construction operation of constructing another machine learning modeling process in the graphical interface by a user while the at least one step is running;

In response to the constructing operation, the another machine learning modeling process is constructed while the at least one step is running.

8. The method of claim 2, wherein the step of receiving a user's modification operation for modifying the machine learning modeling process while running the at least one step comprises:

Receiving a selection operation of a user on a node in the directed acyclic graph while the at least one step is running;

responsive to the selection operation, displaying a control for configuring a configuration item of a step corresponding to the selected node to a user;

and receiving input operation of a user on the displayed control.

9. The method of claim 2, wherein the method further comprises:

And displaying at least one control for displaying at least one output element of the step corresponding to the node around the selected node in response to the selection operation, and displaying the output result of the output element corresponding to the selected control to the user in response to the selection operation of one of the at least one control by the user.

10. The method of claim 2, wherein the step of receiving a user's modification operation for modifying the machine learning modeling process while running the at least one step comprises:

Displaying a node list in a predetermined area of the graphical interface while the at least one step is running, and receiving a user operation of selecting and dragging a node from the node list to connect to a node in the directed acyclic graph;

and/or, while running the at least one step, in response to a user operation for one connection point of one node in the directed acyclic graph, recommending to a user the node and/or the node combination to which the node can be connected through the connection point, and receiving an operation that the user selects one node or one node combination from the recommended nodes and/or node combinations to connect to the connection point.

11. The method of claim 7, wherein the method further comprises:

In response to a user operation for running at least one step in the other machine learning modeling process, running the at least one step in the other machine learning modeling process while running the at least one step in the machine learning modeling process;

Receiving a modification operation of a user for modifying the other machine learning modeling process while running the at least one step of the other machine learning modeling process;

in response to the modifying operation, modifying the other machine learning modeling process,

Wherein when running to the modified portion of the other machine learning modeling process, the modified machine learning modeling process is based on running.

12. A system for constructing a machine learning modeling process, comprising:

Display means for displaying the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process;

Running means for running at least one step in the machine learning modeling process in response to a user operation for running the at least one step, wherein the at least one step includes at least one of data import, data stitching, data splitting, feature extraction, model training, model testing, and model evaluation;

Construction means for receiving a modification operation by a user for modifying the machine learning modeling process while the execution means is executing the at least one step, and modifying the machine learning modeling process in response to the modification operation,

Wherein when the running means runs to the modified part of the machine learning modeling process, running based on the modified machine learning modeling process;

Wherein the modifying operation comprises a configuration operation for modifying a configuration of the step currently being operated or a configuration of a step upstream of the step currently being operated, wherein in response to the configuration operation, the operating means stops operating the step currently being operated, and the constructing means modifies the configuration of the step for which the configuration operation is directed,

Wherein when a user operation for executing at least one step in the machine learning modeling process is received again, the execution means starts to execute from the most upstream modified step,

Wherein when the operating means operates to the modified step, the modified step is operated based on the modified configuration.

13. The system of claim 12, wherein the display device displays a directed acyclic graph representing the constructed machine learning modeling process in a graphical interface for constructing the machine learning modeling process, wherein nodes in the directed acyclic graph correspond one-to-one to steps in the machine learning modeling process.

14. The system of claim 12, wherein the modifying operation comprises a configuration operation for modifying a configuration of a step downstream of the step currently being operated, wherein, in response to the configuration operation, the constructing means modifies the configuration of the step for which the configuration operation is intended while the operating means is operating the at least one step,

15. The system of claim 12, wherein the modifying operation includes a structure adjustment operation for adjusting a structure downstream of a step currently being executed in the machine learning modeling process, wherein, in response to the structure adjustment operation, the constructing means adjusts the structure of the machine learning modeling process while the executing means executes the at least one step,

Wherein when the running means is run to a part of the structure to which the machine learning modeling process is adapted, the running means is run in accordance with the adapted structure.

16. The system of claim 12, wherein the modifying operation includes a structure adjustment operation for adjusting a structure of an upstream structure of a step currently being run in the machine learning modeling process, wherein in response to the structure adjustment operation, the running means stops running the step currently being run, and the constructing means adjusts the structure of the machine learning modeling process,

Wherein when a user operation for executing at least one step in the machine learning modeling process is received again, the executing means starts executing from a portion of the structure that is adjusted furthest upstream in the machine learning modeling process,

17. The system of claim 15 or 16, wherein the structure adjustment operation comprises at least one of an operation to add a step to a machine learning modeling process, an operation to delete a step in a machine learning modeling process, and an operation to change a logical relationship between steps in a machine learning modeling process.

18. The system of claim 12, wherein the build device receives a build operation by a user to build another machine learning modeling process in the graphical interface while the at least one step is being run by the running device, and builds the another machine learning modeling process while the at least one step is being run in response to the build operation.

19. The system of claim 13, wherein the constructing means receives a user selection operation of a node in the directed acyclic graph while the operating means is operating the at least one step, displays a control for configuring a configuration item of the step corresponding to the selected node to the user in response to the selection operation, and receives an input operation of the user on the displayed control.

20. The system according to claim 13, wherein the constructing means receives a user's selection operation of a node in the directed acyclic graph while the operating means operates the at least one step, displays at least one control for presenting at least one output element of the step corresponding to the node, respectively, around the selected node in response to the selection operation, and presents an output result of the output element corresponding to the selected control to the user in response to the user's selection operation of one of the at least one control.

21. The system of claim 13, wherein the constructing means displays a list of nodes in a predetermined area of the graphical interface while the running means runs the at least one step and receives a user operation to select and drag a node from the list of nodes to connect to a node in the directed acyclic graph;

and/or the construction means, while the operation means is running the at least one step, in response to a user operation for one connection point of one node in the directed acyclic graph, recommends to the user the node and/or the node combination to which the node can be connected through the connection point, and receives an operation that the user selects one node or one node combination from the recommended node and/or node combination to connect to the connection point.

22. The system of claim 18, wherein,

Running means, in response to a user operation for running at least one step in the other machine learning modeling process, for running the at least one step in the other machine learning modeling process while running the at least one step in the machine learning modeling process;

the constructing means receives a modification operation by the user for modifying the other machine learning modeling process while the operating means operates the at least one step in the other machine learning modeling process, and modifies the other machine learning modeling process in response to the modification operation,

Wherein when the running means runs to the modified part of the other machine learning modeling process, the running is based on the modified machine learning modeling process.

23. A computer readable medium having recorded thereon a computer program for executing the method for constructing a machine learning modeling process according to any of claims 1 to 11.

24. A computing device comprising a storage means and a processor, wherein the storage means has stored therein a set of computer-executable instructions that, when executed by the processor, perform the method for constructing a machine learning modeling process of any of claims 1 to 11.