CN116956294A - Code attack detection methods, systems, equipment and media applied to training models - Google Patents

Abstract

The application discloses a code attack detection method, a system, equipment and a medium applied to a training model, which mainly relate to the technical field of code detection and are used for solving the problems that the existing method is invalid for a trigger specific to a tag and is also invalid for a trigger unknown to the tag, and the code attack detection cannot be effectively realized. Comprising the following steps: obtaining model labels generated by training; determining a model label which can cause the training module to perform error classification after the modification of the preset variable data as an attack target label; the absolute intermediate level of the obtained model tag is further determined, an abnormal value in the model tag is further determined, and the model tag corresponding to the abnormal value is determined to be an attack target tag; determining whether loadable data exists or not through a preset code detection statement; when loadable data exists, determining that a model tag corresponding to the loadable data exists as an attack target tag; and acquiring an active filter corresponding to the attack target label, and taking the active filter as an input filter of the training model.

Description

Code attack detection methods, systems, equipment and media applied to training models

Technical field

The present application relates to the field of code detection technology, and in particular to a code attack detection method, system, equipment and medium applied to training models.

Background technique

In recent years, with the gradual development of health and medical big data centers, more and more data have entered the application layer stage. In artificial intelligence application scenarios of health and medical big data, users often only have permission to use data for model training. After the model training is completed, the model can be used by users after being reviewed, and after the data completes training the model, it returns to the big data center. In the summary of the above application scenarios, the review stage of the model is the key and difficult point, which involves possible code attacks in the medical model.

Depending on the impact, code attacks can be summarized as: code poisoning attacks, outsourcing attacks, pre-training attacks, data collection attacks, collaborative learning attacks, and post-deployment attacks. Currently, code attack detection and defense methods mainly fall into the following categories: ① Defense methods based on pruning: resist by pruning the neurons corresponding to coding triggers; ② Defense methods based on saliency maps: This method first calculates each The saliency map of each picture is used to locate the trigger based on the areas with the same saliency between different pictures.

However, the above-mentioned pruning-based defense methods are ineffective against tag-specific triggers, and the saliency map-based defense methods are also ineffective against tag-specific triggers, thus failing to effectively achieve the purpose of code attack detection and defense.

Contents of the invention

In view of the above-mentioned shortcomings of the existing technology, this application provides a code attack detection method, system, equipment and medium for training models to solve the problem that the existing method is ineffective for tag-specific triggers and triggers for unknown tags. The device is also invalid and cannot effectively achieve the purpose of code attack detection.

In the first aspect, this application provides a code attack detection method applied to a training model. The method includes: obtaining a trained training model to obtain model labels generated by training; through preset triggers and preset variable data, It is determined that the model labels that can cause the training module to misclassify after modification of the preset variable data are attack target labels; based on the absolute median difference technology, the absolute median difference of the model labels is obtained, and then the outliers in the model labels are determined. Determine the model label corresponding to the outlier as the attack target label; use the preset code detection statement to detect the model label to determine whether there is loadable data; when there is loadable data, determine that there is a model label corresponding to the loadable data. is the attack target label; obtain the active filter corresponding to the attack target label, and use the active filter as the input filter of the training model.

Further, after detecting the model label through the preset code detection statement to determine whether loadable data exists, the method also includes: when loadable data exists, obtain the replacement corresponding to the loadable data through the backend editing interface data.

Further, before determining through the preset trigger and preset variable data that the model label that can cause the training module to misclassify after modification of the preset variable data is the attack target label, the method includes: obtaining the preset variable through the preset interface Data; among them, the preset variable data includes model tag names and variable values.

Further, through the preset trigger and preset variable data, it is determined that the model label that can cause the training module to misclassify after modification of the preset variable data is the attack target label, which specifically includes: using the preset trigger, changing the model label name The corresponding model label increases or decreases the variable value to obtain the change range of the model label; determine whether the change range of the model label intersects with the model label danger range preset in the preset trigger. When there is an intersection area, determine Model labels that can cause the training module to misclassify after modifying the preset variable data are attack target labels.

In the second aspect, this application provides a code attack detection system applied to a training model. The system includes: an acquisition module, used to acquire a trained training model to obtain model labels generated by training; a determination module, used to pass Preset trigger and preset variable data, determine the model label that can cause the training module to misclassify after modifying the preset variable data as the attack target label; based on the absolute median difference technology, obtain the absolute median difference of the model label , and then determine the outliers in the model labels, and determine the model labels corresponding to the outliers as attack target labels; detect the model labels through preset code detection statements to determine whether there is loadable data; when there is loadable data , it is determined that the model label corresponding to the loadable data is the attack target label; the filter acquisition module is used to obtain the active filter corresponding to the attack target label, so as to use the active filter as the input filter of the training model.

Further, the determination module includes a replacement unit, which is used to obtain replacement data corresponding to the loadable data through the back-end editing interface when there is loadable data.

Further, the determination module includes an acquisition unit for obtaining preset variable data through a preset interface; wherein the preset variable data includes model tag names and variable values.

Further, the determination module includes a trigger unit, which is used to increase or decrease the variable value of the model label corresponding to the model label name through a preset trigger to obtain the change range of the model label; determine whether the change range of the model label is consistent with the predetermined change range. It is assumed that the dangerous range of the model labels preset in the trigger has an intersecting area. When the intersecting area exists, the model label that can cause the training module to be misclassified after modifying the preset variable data is determined to be the attack target label.

In a third aspect, this application provides a code attack detection device applied to a training model. The device includes: a processor; and a memory on which executable code is stored. When the executable code is executed, the processor executes: A code attack detection method applying any of the above to the trained model.

In a fourth aspect, the present application provides a non-volatile computer storage medium on which computer instructions are stored. When executed, the computer instructions implement any one of the above code attack detection methods applied to training models.

Those skilled in the art can understand that this application has at least the following beneficial effects:

This application realizes multi-faceted detection of whether there is an attack target label by traversing model label trigger detection, absolute median difference technology and preset code detection statements. For a training model that does not know the label, it is still possible to obtain the attack target label. By obtaining the active filter corresponding to the attack target label, the active filter can be used as the input filter of the training model. After obtaining the label with tool risk (attack target label), obtain an active filter that can filter the risk input corresponding to the attack target label, and use the active filter as the input filter of the training model to filter out the external input data attack tool. Risk tag (attack target tag) risk.

Description of the drawings

Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which:

Figure 1 is a flow chart of a code attack detection method applied to training models provided by an embodiment of the present application.

Figure 2 is a schematic diagram of the internal structure of a code attack detection system applied to training models provided by an embodiment of the present application.

Figure 3 is a schematic diagram of the internal structure of a code attack detection device used in training models provided by an embodiment of the present application.

Detailed ways

Those skilled in the art should understand that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be realized through the preferred embodiments. The preferred embodiments are only used to explain the present disclosure. Technical principles are not used to limit the scope of protection of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by those of ordinary skill in the art without exerting creative efforts should still fall within the protection scope of the present disclosure.

It should also be noted that the terms "comprises," "comprises" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements not only includes those elements, but also includes Other elements are not expressly listed or are inherent to the process, method, article or equipment. Without further limitation, an element qualified by the statement "comprises a..." does not exclude the presence of additional identical elements in the process, method, good, or device that includes the element.

The technical solutions proposed in the embodiments of the present application will be described in detail below through the accompanying drawings.

The embodiment of the present application also provides a code attack detection method applied to the training model. As shown in Figure 1, the method provided by the embodiment of the present application mainly includes the following steps:

Step 110: Obtain the trained training model to obtain the model label generated by training.

It should be noted that the training model can be a deep neural network model. The process of obtaining model labels can be obtained by those skilled in the art based on existing technologies, and is not limited in this application.

Step 120: Through the preset trigger and preset variable data, determine that the model label that can cause the training module to misclassify after modifying the preset variable data is the attack target label; based on the absolute median difference technology, use the obtained model label The absolute median difference is used to determine the outliers in the model labels, and the model labels corresponding to the outliers are determined as attack target labels; through preset code detection statements, the model labels are detected to determine whether there is loadable data; to determine if there is When data can be loaded, it is determined that the model label corresponding to the loadable data is the attack target label.

It should be noted that the absolute median difference technique is an existing outlier detection and is used to detect outliers. The preset code detection statement is used to detect code such as loading model tags. The preset code detection statement can be specifically tf.data.Dataset, map(func), etc. Loadable data is data that can be loaded. Specifically, models that can be loaded by preset code detection statements are labeled as loadable data.

In addition, this application can provide an interface to replace loadable data with unloadable data. The specific process can be: when loadable data exists, obtain the replacement data corresponding to the loadable data through the back-end editing interface.

In order to effectively update the preset variable data and improve the accuracy of the preset trigger, this application can determine the model that can cause the training module to misclassify after modifying the preset variable data through the preset trigger and the preset variable data. Before the label is an attack target label, the preset variable data (initial or changed) is obtained through the preset interface; where the preset variable data includes the model label name and variable value.

Among them, through the preset trigger and preset variable data, it is determined that the model label that can cause the training module to misclassify after modification of the preset variable data is the attack target label. Specifically, it can be:

Through the preset trigger, increase or decrease the variable value of the model label corresponding to the model label name to obtain the change range of the model label; determine whether the change range of the model label is consistent with the preset dangerous range of the model label in the preset trigger. Intersection area. When there is an intersection area, determine the model label that can cause the training module to misclassify after modifying the preset variable data as the attack target label.

It should be noted that the preset trigger can increase or decrease the variable value of the model label corresponding to the model label name to obtain the change range of the model label; determine whether the change range of the model label is consistent with the model preset in the preset trigger. There is an intersecting area in the label danger range. When there is an intersecting area, it is determined that the model label that can cause the training module to misclassify after modifying the preset variable data is a device or device with an attack target label.

Step 130: Obtain the active filter corresponding to the attack target label, and use the active filter as an input filter for the training model.

It should be noted that the process of obtaining the active filter corresponding to the attack target label can be obtained through the preset filter acquisition interface.

In addition, Figure 2 shows a code attack detection system applied to training models provided by an embodiment of the present application. As shown in Figure 2, the system provided by the embodiment of this application mainly includes:

The system obtains the trained training model through the obtaining module 210 to obtain model labels generated by training.

It should be noted that the obtaining module 210 can be any feasible device or device that can obtain a training model and obtain a model label from the model.

The determination module 220 in the system determines that the model label that can cause the training module to misclassify after modification of the preset variable data is the attack target label through the preset trigger and preset variable data; based on the absolute median difference technology, to obtain The absolute median difference of the model labels is used to determine the outliers in the model labels, and the model labels corresponding to the outliers are determined to be attack target labels; through preset code detection statements, the model labels are detected to determine whether there is loadable data; When loadable data exists, the model label corresponding to the loadable data is determined to be the attack target label.

It should be noted that the determination module 220 may be any feasible device or device capable of filtering out attack target tags from model tags.

In addition, this application can provide an interface to replace loadable data with unloadable data. The specific process may be: when loadable data exists, the replacement unit 221 in the determination module 220 obtains the corresponding loadable data through the back-end editing interface. Replace data.

In order to effectively update the preset variable data and improve the accuracy of the preset trigger, the acquisition unit 222 in the determination module 220 obtains the preset variable data through the preset interface; where the preset variable data includes model tag names and variable values.

Among them, through the preset trigger and preset variable data, it is determined that the model label that can cause the training module to misclassify after modification of the preset variable data is the attack target label. Specifically, it can be:

The trigger unit 223 in the determination module 220 uses a preset trigger to increase or decrease the variable value of the model label corresponding to the model label name to obtain the change range of the model label; determine whether the change range of the model label is consistent with the preset trigger There is an intersecting area in the dangerous range of the preset model labels. When there is an intersecting area, the model label that can cause the training module to be misclassified after modifying the preset variable data is determined to be the attack target label.

The filter acquisition module 230 is used to acquire the active filter corresponding to the attack target label, so as to use the active filter as an input filter for the training model.

The filter acquisition module 230 is any feasible device or device that can acquire active filters and use active filters as input filters for training models.

The above are the method embodiments in this application. Based on the same inventive concept, the embodiments of this application also provide a code attack detection device applied to training models. As shown in Figure 3, the device includes: a processor; and a memory on which executable code is stored. When the executable code is executed, the processor executes one of the codes applied to training the model in the above embodiment. Attack detection methods.

Specifically, the server side obtains the trained training model to obtain the model label generated by the training; through the preset trigger and the preset variable data, it determines the model label that can cause the training module to misclassify after modifying the preset variable data. is the attack target label; based on the absolute median difference technology, the absolute median difference of the model labels is obtained, and then the outliers in the model labels are determined, and the model label corresponding to the outlier is determined to be the attack target label; statements are detected through the preset code , detect the model label to determine whether there is loadable data; when there is loadable data, it is determined that the model label corresponding to the loadable data is the attack target label; obtain the active filter corresponding to the attack target label, so as to convert the active filter serves as an input filter for training the model.

In addition, embodiments of the present application also provide a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, the above-mentioned method for training a model is implemented. Code attack detection methods.

So far, the technical solutions of the present disclosure have been described in conjunction with the foregoing embodiments. However, those skilled in the art can easily understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent substitutions, improvements, etc. made within the concept and/or technical principles will fall within the protection scope of the present disclosure.

Claims (10)

1. A method for detecting a code attack applied to a training model, the method comprising:

obtaining a trained training model to obtain a model label generated by training;

determining a model tag which can cause the training module to perform error classification after modification of the preset variable data as an attack target tag through the preset trigger and the preset variable data; based on an absolute medium-level difference technology, determining an abnormal value in the model label by using the acquired absolute medium-level difference of the model label, and determining the model label corresponding to the abnormal value as an attack target label; detecting the model tag through a preset code detection statement to determine whether loadable data exists or not; when loadable data exists, determining that a model tag corresponding to the loadable data exists as an attack target tag;

and acquiring an active filter corresponding to the attack target label, and taking the active filter as an input filter of the training model.

2. The method for detecting a code attack applied to a training model according to claim 1, wherein after detecting a model tag by a preset code detection statement to determine whether loadable data exists, the method further comprises:

when loadable data exists, acquiring replacement data corresponding to the loadable data through a back-end editing interface.

3. The code attack detection method applied to a training model according to claim 1, wherein before determining, through a preset trigger and preset variable data, that a model tag capable of causing a training module to perform an erroneous classification after modification of the preset variable data is an attack target tag, the method comprises:

acquiring preset variable data through a preset interface; the preset variable data comprises a model tag name and a variable value.

4. The method for detecting a code attack applied to a training model according to claim 3, wherein the model tag which can cause the training module to perform error classification after modification of the preset variable data is determined to be an attack target tag through a preset trigger and preset variable data, and specifically comprises:

the method comprises the steps that through a preset trigger, a model label corresponding to a model label name is increased or decreased in variable value, so that the change range of the model label is obtained;

determining whether an intersection area exists between the change range of the model tag and the dangerous range of the model tag preset in the preset trigger, and determining that the model tag which can cause the training module to perform error classification after the preset variable data modification is an attack target tag when the intersection area exists.

5. A code attack detection system for application to a training model, the system comprising:

the obtaining module is used for obtaining a trained training model so as to obtain a model label generated by training;

the determining module is used for determining that the model tag which can cause the training module to perform error classification after the modification of the preset variable data is an attack target tag through the preset trigger and the preset variable data; based on an absolute medium-level difference technology, determining an abnormal value in the model label by using the acquired absolute medium-level difference of the model label, and determining the model label corresponding to the abnormal value as an attack target label; detecting the model tag through a preset code detection statement to determine whether loadable data exists or not; when loadable data exists, determining that a model tag corresponding to the loadable data exists as an attack target tag;

the filter acquisition module is used for acquiring an active filter corresponding to the attack target label so as to take the active filter as an input filter of the training model.

6. The code attack detection system applied to a training model according to claim 5, wherein the determination module comprises a replacement unit,

and acquiring the replacement data corresponding to the loadable data through the back-end editing interface when the loadable data exists.

7. The code attack detection system applied to a training model according to claim 5, wherein the determining module comprises an acquiring unit,

the method comprises the steps of obtaining preset variable data through a preset interface; the preset variable data comprises a model tag name and a variable value.

8. The code attack detection system applied to a training model according to claim 7, wherein the determining module comprises a trigger unit,

the method comprises the steps that a model label corresponding to a model label name is increased or decreased by a variable value through a preset trigger, so that the change range of the model label is obtained; determining whether an intersection area exists between the change range of the model tag and the dangerous range of the model tag preset in the preset trigger, and determining that the model tag which can cause the training module to perform error classification after the preset variable data modification is an attack target tag when the intersection area exists.

9. A code attack detection device for application to a training model, the device comprising:

a processor;

and a memory having executable code stored thereon that, when executed, causes the processor to perform a code attack detection method applied to a training model as claimed in any of claims 1-4.

10. A non-transitory computer storage medium having stored thereon computer instructions which, when executed, implement a code attack detection method applied to a training model according to any of claims 1-4.