CN104008336A - ShellCode detecting method and device - Google Patents

Abstract

The present invention provides a ShellCode detection method and device, comprising: establishing a ShellCode instruction sequence feature library comprising a ShellCode feature sequence; loading a file to be detected, analyzing the file to be detected to form an analysis file; performing an analysis on the analysis file Simulated execution and analysis of instructions, recording suspicious instruction sequences and comparing them with the ShellCode feature sequences in the ShellCode instruction sequence feature library, judging whether ShellCode is included in the analysis file; outputting detection results. The invention can detect the ShellCode before executing the ShellCode, prevents the impact of the ShellCode on the system, and prevents the possibility of malicious tampering in time.

Description

A ShellCode detection method and device

technical field

The invention relates to the field of network security, in particular to a ShellCode detection method and device.

Background technique

At present, buffer overflow is a main springboard for various system attacks and gaining control of the system. To implement buffer overflow attacks, ShellCode must be used, and the attack data must contain the corresponding ShellCode. Therefore, strengthening the detection of attacks such as buffer overflow is an important research content in the field of information security. At the same time, currently widely used software such as Office, Adobe Reader and other document processing software has become a springboard for many ShellCode attacks. Many attackers hide ShellCode in these commonly used documents such as DOC and PDF to attack.

The current detection of ShellCode is mainly divided into static detection and dynamic detection. Static ShellCode detection technology generally uses pattern matching and static disassembly technology. By scanning and analyzing the data, according to the feature code in the ShellCode knowledge base, it is possible to identify possible existence of ShellCode. Its characteristic is that the detection speed is fast, but it has the problem that it is not well targeted and cannot detect unknown ShellCode. At the same time, static binary analysis does not actually execute instructions, and cannot automatically capture the control transfer of instructions, and relies entirely on manual analysis of the control flow graph of the program.

The dynamic ShellCode detection technology simulates the execution of the data flow as an executable code. By actually running the program, it can automatically capture the control transfer of the program instruction, monitor and analyze the program's running data access, and is not affected by the code obfuscation technology. Influence. However, the existing ShellCode detection technology based on memory search is to detect after the ShellCode has been loaded into the memory, it is difficult to prevent the changes made by the ShellCode, and it is difficult to accurately and effectively search the memory when the amount of data is large; Virtually executed ShellCode detection technology requires the support of corresponding virtual machine technology. In the case of a large amount of data, a large number of virtual machines need to be deployed, and many current ShellCode codes have the function of virtual machine confrontation.

In addition, most of the currently implemented technologies are aimed at the analysis of network data flow, but there is insufficient support for the detection of ShellCode attacking local programs, and the damage that ShellCode has already caused to the system cannot be avoided.

Contents of the invention

(1) Technical problems to be solved

The invention provides a ShellCode detection method and device to solve the technical problem that the existing technology can only detect after the ShellCode is running, which cannot avoid the existing damage to the system and the confusion technology makes it difficult to statically detect the ShellCode.

(2) Technical solution

In order to solve the problems of the technologies described above, the invention provides a ShellCode detection method, comprising:

Establish a ShellCode instruction sequence feature library containing the ShellCode feature sequence;

Loading the file to be detected, analyzing the file to be detected to form an analysis file;

Carry out the simulated execution and analysis of instructions to the analysis file, record the suspicious instruction sequence and compare it with the ShellCode feature sequence in the ShellCode instruction sequence feature library, and judge whether ShellCode is included in the analysis file;

Output the detection result.

Further, said parsing the file to be detected to form the parsing file includes:

Parse according to the type of the file to be detected, and convert it into a parsing file in a unified format.

Further, the document types include:

One or more of DOC, DOCX, XLS, XLSX, PPT, PPTX, PDF.

Further, the simulated execution and analysis of instructions are performed on the analysis file, the suspicious instruction sequence is recorded and compared with the ShellCode feature sequence in the ShellCode instruction sequence feature library, and whether the ShellCode is included in the analysis file is judged, including :

Step S1: Initialize the step size of the suspicious instruction sequence;

Step S2: Using the parsed file as a binary string to simulate execution and analysis of instructions, when a suspicious instruction is detected, execute step S3, otherwise, execute step S6;

Step S3: adding 1 to the step size of the suspicious instruction sequence;

Step S4: compare whether the step size of the suspicious instruction sequence reaches the threshold value of the ShellCode feature sequence in the ShellCode instruction sequence feature library, if it reaches the threshold value, execute step S5, otherwise, execute step S6;

Step S5: Determine that the parsed file data contains ShellCode, and record the alarm information;

Step S6: Judging whether the detection of all data in the analysis file is completed, if all data detection is completed, it is determined that the analysis file does not contain ShellCode, otherwise, return to step S2.

Further, the method also includes:

The ShellCode feature sequence in the ShellCode instruction sequence feature library is updated by using the ShellCode detection result.

On the other hand, the present invention also provides a ShellCode detection device, comprising: a feature library module, a scheduling module, a file parsing module, a simulation analysis module and a result output module, the scheduling module is connected to the file parsing module, and the simulation The analysis module is respectively connected with the feature library module, the file parsing module and the result output module, wherein:

A feature library module, used to set up a ShellCode instruction sequence feature library that includes a ShellCode feature sequence;

A scheduling module for loading files to be detected;

A file analysis module, configured to analyze the file to be detected to form an analysis file;

The simulation analysis module is used to simulate execution and analysis of instructions to the analysis file, record suspicious instruction sequences and compare them with the ShellCode feature sequence in the ShellCode instruction sequence feature library, and determine whether ShellCode is included in the analysis file;

The result output module is used to output the detection result.

Further, the file parsing module is also used for:

Parse according to the type of the file to be detected, and convert it into a parsing file in a unified format.

Further, the simulation analysis module is also used to perform the following steps:

Step S1: Initialize the step size of the suspicious instruction sequence;

Step S2: Using the parsed file as a binary string to simulate execution and analysis of instructions, when a suspicious instruction is detected, execute step S3, otherwise, execute step S6;

Step S3: adding 1 to the step size of the suspicious instruction sequence;

Step S4: compare whether the step size of the suspicious instruction sequence reaches the threshold value of the ShellCode feature sequence in the ShellCode instruction sequence feature library, if yes, perform step S5, otherwise, perform step S6;

Step S5: Determine that the parsed file data contains ShellCode, and record the alarm information;

Step S6: Judging whether the detection of all data in the analysis file is completed, if all data detection is completed, it is determined that the analysis file does not contain ShellCode, otherwise, return to step S2.

Further, the simulation analysis module is:

An instruction simulator capable of executing a specified step size, which can simulate registers and/or key components required for FPU instruction execution.

Further, the device also includes:

An update module, connected with the feature library module and the result output module, is used to update the ShellCode feature sequence in the ShellCode instruction sequence feature library by using the ShellCode detection result.

(3) Beneficial effects

It can be seen that in the ShellCode detection method and device provided in the embodiment of the present invention, document analysis and instruction simulation technology can be used to analyze the file to be detected, and then use the simulation instruction to judge whether there is a suspicious instruction sequence, and compare it with the pre-established instruction Sequence feature library is compared to determine whether ShellCode is included. The ShellCode detection method in the embodiment of the present invention can detect the ShellCode before the execution of the ShellCode, which prevents the impact of the ShellCode on the system and prevents the possibility of malicious tampering in time. Moreover, the embodiment of the present invention can analyze the instruction sequence without deploying a complete virtual machine, which reduces deployment requirements and costs.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is the basic flowchart schematic diagram of the ShellCode detection method of the embodiment of the present invention;

Fig. 2 is a preferred flowchart of the ShellCode detection method of the embodiment of the present invention;

Fig. 3 is a schematic diagram of the ShellCode judging process of the embodiment of the present invention;

4 is a schematic diagram of the basic structure of a ShellCode detection device according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of a preferred structure of a ShellCode detection device according to an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

ShellCode is the execution carrier of vulnerability attack. A complete vulnerability attack requires the cooperation of vulnerability utilization technology and ShellCode to achieve the desired attack effect. In view of this problem, the embodiment of the present invention firstly provides a ShellCode detection method, see Figure 1, including:

Step 101: Establish a ShellCode instruction sequence feature library containing the ShellCode feature sequence.

Step 102: Load the file to be detected, analyze the file to be detected, and form an analysis file.

Step 103: Simulate execution and analysis of instructions on the analysis file, record suspicious instruction sequences and compare them with the ShellCode feature sequences in the ShellCode instruction sequence feature library, and determine whether the analysis file contains ShellCode.

Step 104: output the detection result.

It can be seen that in the ShellCode detection method provided by the embodiment of the present invention, document analysis and instruction simulation technology can be used to analyze the file to be detected, and then use the simulation instruction to judge whether there is a suspicious instruction sequence, and compare it with the pre-established instruction sequence characteristics. Libraries are compared to determine whether ShellCode is included. The ShellCode detection method in the embodiment of the present invention can detect the ShellCode before the execution of the ShellCode, which prevents the impact of the ShellCode on the system and prevents the possibility of malicious tampering in time. Moreover, the embodiment of the present invention can analyze the instruction sequence without deploying a complete virtual machine, which reduces deployment requirements and costs.

The embodiment of the present invention can identify various file types. Preferably, analyzing the file to be detected, and forming the parsed file may include: parsing according to the type of the file to be detected, and converting it into a parsed file in a unified format.

Preferably, the document type may include: one or more of DOC, DOCX, XLS, XLSX, PPT, PPTX, PDF.

Preferably, carry out the simulated execution and analysis of instructions to the analysis file, record the suspicious instruction sequence and compare it with the ShellCode feature sequence in the ShellCode instruction sequence feature library, and determine whether ShellCode is included in the analysis file can specifically include the following step:

Step S1: Initialize the step size of the suspicious instruction sequence;

Step S2: Using the parsed file as a binary string to simulate execution and analysis of instructions, when a suspicious instruction is detected, execute step S3, otherwise, execute step S6;

Step S3: adding 1 to the step size of the suspicious instruction sequence;

Step S4: compare whether the step size of the suspicious instruction sequence reaches the threshold value of the ShellCode feature sequence in the ShellCode instruction sequence feature library, if it reaches the threshold value, execute step S5, otherwise, execute step S6;

Step S5: Determine that the parsed file data contains ShellCode, and record the alarm information;

Step S6: Judging whether the detection of all data in the analysis file is completed, if all data detection is completed, it is determined that the analysis file does not contain ShellCode, otherwise, return to step S2.

The method in the embodiment of the present invention can also update the ShellCode instruction sequence feature library in time, and can continuously adjust and correct errors to avoid misjudgment next time. Preferably, the method can also include: using the ShellCode detection result to update all Update the ShellCode feature sequence in the above ShellCode command sequence feature library.

Taking the specific ShellCode detection process as an example below, to describe the specific implementation process of the embodiment of the present invention in detail, see Figure 2:

Step 201: Establish a ShellCode instruction sequence feature library containing the ShellCode feature sequence.

In this step, first a ShellCode instruction sequence feature library needs to be established, which contains known ShellCode feature sequences.

Step 202: Load the file to be detected, analyze the file to be detected, and form an analysis file.

In this step, the file to be detected is first loaded, and then the type of the file is identified, and analyzed according to the identification result to form an analysis file in a unified format.

Step 203: Carry out instruction simulation execution and analysis on the parsing file, and judge whether it contains ShellCode.

In this step, an instruction simulator similar to LibEmu is invoked, and by simulating the execution of assembly instructions, the suspicious instruction sequence is recorded and compared with the signature sequence in the ShellCode instruction sequence signature library, and whether the file contains ShellCode is analyzed.

The specific steps are shown in Figure 3, wherein, when initializing the state machine, set the step length of the recorded suspicious instruction sequence to 0; when judging whether it is a suspicious instruction, regard the parsed document data as a binary string, and use the instruction simulator Simulate registers, FPU, etc. to simulate the execution of instructions to analyze the instructions and find out whether they contain suspicious ShellCode; when suspicious instructions are detected, the status of the state machine is updated, and the step size of the suspicious instruction of the state machine is increased by 1.

Step 204: Output the detection result and update the ShellCode instruction sequence feature library.

In this step, the final detection result is output, and the ShellCode instruction sequence feature library is updated and adjusted according to the result, so as to avoid misjudgment next time.

So far, the whole process of the ShellCode detection method of the embodiment of the present invention is completed.

The embodiment of the present invention also provides a ShellCode detection device, referring to FIG. 4 , including: a feature library module 401, a scheduling module 402, a file parsing module 403, a simulation analysis module 404, and a result output module 405, wherein:

Feature library module 401, for setting up the ShellCode instruction sequence feature library that comprises ShellCode feature sequence;

Scheduling module 402, for loading the file to be detected;

A file analysis module 403, configured to analyze the file to be detected to form an analysis file;

The simulation analysis module 404 is used for the simulation execution and analysis of the instruction to the analysis file, records the suspicious instruction sequence and compares it with the ShellCode feature sequence in the ShellCode instruction sequence feature library, and judges whether the analysis file contains ShellCode ;

The result output module 405 is configured to output the detection result.

Preferably, the file parsing module 403 can also be used for parsing according to the type of the file to be detected, and converting it into a parsing file in a unified format.

Preferably, the simulation analysis module 404 is also configured to perform the following steps:

Step S1: Initialize the step size of the suspicious instruction sequence;

Step S2: Using the parsed file as a binary string to simulate execution and analysis of instructions, when a suspicious instruction is detected, execute step S3, otherwise, execute step S6;

Step S3: adding 1 to the step size of the suspicious instruction sequence;

Step S4: compare whether the step size of the suspicious instruction sequence reaches the threshold value of the ShellCode feature sequence in the ShellCode instruction sequence feature library, if yes, perform step S5, otherwise, perform step S6;

Step S5: Determine that the parsed file data contains ShellCode, and record the alarm information;

Step S6: Judging whether the detection of all data in the analysis file is completed, if all data detection is completed, it is determined that the analysis file does not contain ShellCode, otherwise, return to step S2.

Preferably, the simulation analysis module 303 may be: an instruction simulator capable of simulating components such as registers and FPU, and executing a specified step size.

Preferably, the device can also include: an update module 501, see Figure 5, which is connected to the feature library module 401 and the result output module 405 respectively, for updating the ShellCode feature sequence in the ShellCode instruction sequence feature library using the ShellCode detection result .

It can be seen that the embodiments of the present invention have the following beneficial effects:

In a ShellCode detection method and device provided in the embodiments of the present invention, document analysis and command simulation technology can be used to analyze the document to be detected, and then use the simulated command to judge whether there is a suspicious command sequence, and compare it with the pre-established command sequence characteristics Libraries are compared to determine whether ShellCode is included. The ShellCode detection method in the embodiment of the present invention can detect the ShellCode before the execution of the ShellCode, which prevents the impact of the ShellCode on the system and prevents the possibility of malicious tampering in time. Moreover, the embodiment of the present invention can analyze the instruction sequence without deploying a complete virtual machine, which reduces deployment requirements and costs.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (10)

1. A ShellCode detection method is characterized in that, comprising: Establish a ShellCode instruction sequence feature library containing the ShellCode feature sequence; Loading the file to be detected, analyzing the file to be detected to form an analysis file; Carry out the simulated execution and analysis of instructions to the analysis file, record the suspicious instruction sequence and compare it with the ShellCode feature sequence in the ShellCode instruction sequence feature library, and judge whether ShellCode is included in the analysis file; Output the detection result. 2. the ShellCode detection method according to claim 1, is characterized in that, described described to-be-detected file is analyzed, and formation analysis file comprises: Parse according to the type of the file to be detected, and convert it into a parsing file in a unified format. 3. the ShellCode detection method according to claim 2, is characterized in that, described document type comprises: One or more of DOC, DOCX, XLS, XLSX, PPT, PPTX, PDF. 4. the ShellCode detection method according to claim 1, is characterized in that, described parsing file is carried out the simulated execution and the analysis of instruction, records suspicious instruction sequence and with the ShellCode characteristic sequence in the described ShellCode instruction sequence characteristic storehouse Compare and judge whether ShellCode is included in the parsing file, including: Step S1: Initialize the step size of the suspicious instruction sequence; Step S2: Using the parsed file as a binary string to simulate execution and analysis of instructions, when a suspicious instruction is detected, execute step S3, otherwise, execute step S6; Step S3: adding 1 to the step size of the suspicious instruction sequence; Step S4: compare whether the step size of the suspicious instruction sequence reaches the threshold value of the ShellCode feature sequence in the ShellCode instruction sequence feature library, if it reaches the threshold value, execute step S5, otherwise, execute step S6; Step S5: Determine that the parsed file data contains ShellCode, and record the alarm information; Step S6: Judging whether the detection of all data in the analysis file is completed, if all data detection is completed, it is determined that the analysis file does not contain ShellCode, otherwise, return to step S2. 5. according to the ShellCode detection method described in any one in claim 1 to 4, it is characterized in that, described method also comprises: The ShellCode feature sequence in the ShellCode instruction sequence feature library is updated by using the ShellCode detection result. 6. a ShellCode detection device, it is characterized in that, comprises: feature library module, scheduling module, file analysis module, simulation analysis module and result output module, described scheduling module links to each other with described file analysis module, and described simulation analysis module Link to the feature library module, file parsing module and result output module respectively, wherein: A feature library module, used to set up a ShellCode instruction sequence feature library that includes a ShellCode feature sequence; A scheduling module for loading the file to be detected; A file analysis module, configured to analyze the file to be detected to form an analysis file; The simulation analysis module is used to simulate execution and analysis of instructions to the analysis file, record suspicious instruction sequences and compare them with the ShellCode feature sequence in the ShellCode instruction sequence feature library, and determine whether ShellCode is included in the analysis file; The result output module is used to output the detection result. 7. ShellCode detecting device according to claim 6, is characterized in that, described file parsing module is also used for: Parse according to the type of the file to be detected, and convert it into a parsing file in a unified format. 8. ShellCode detection device according to claim 6, is characterized in that, described analog analysis module is also used for performing the following steps: Step S1: Initialize the step size of the suspicious instruction sequence; Step S2: Using the parsed file as a binary string to simulate execution and analysis of instructions, when a suspicious instruction is detected, execute step S3, otherwise, execute step S6; Step S3: adding 1 to the step size of the suspicious instruction sequence; Step S4: compare whether the step size of the suspicious instruction sequence reaches the threshold value of the ShellCode feature sequence in the ShellCode instruction sequence feature library, if yes, perform step S5, otherwise, perform step S6; Step S5: Determine that the parsed file data contains ShellCode, and record the alarm information; Step S6: Judging whether the detection of all data in the analysis file is completed, if all data detection is completed, it is determined that the analysis file does not contain ShellCode, otherwise, return to step S2. 9. ShellCode detecting device according to claim 8, is characterized in that, described analog analysis module is: An instruction simulator capable of executing a specified step size, which can simulate registers and/or key components required for FPU instruction execution. 10. The ShellCode detection device according to any one of claims 6 to 9, wherein the device further comprises: An update module, connected with the feature library module and the result output module, is used to update the ShellCode feature sequence in the ShellCode instruction sequence feature library by using the ShellCode detection result.