Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the following detailed description, reference is made to the accompanying drawings that form a part hereof and in which is shown by way of illustration specific embodiments of the application. In the drawings, like numerals describe substantially similar components throughout the different views. Various specific embodiments of the present application are described in sufficient detail below to enable those skilled in the art to practice the teachings of the present application. It is to be understood that other embodiments may be utilized and structural, logical or electrical changes may be made to the embodiments of the present application.
Fig. 1 is a schematic block diagram of a vulnerability detection system provided according to an embodiment of the present invention. Referring to fig. 1, the vulnerability detection system includes a test end browser 1, an optional manual detection tool 2, an automatic detection tool 3, and an item detection management module 4, where the item detection management module 4 is connected to the test end browser 1, the manual detection tool 2, and the automatic detection tool 3. The test end browser 1 is a browser used by a safety engineer, and a designated proxy port P is set by configuring a browser proxy server on a computer used by the safety engineer, so that the test end browser 1 is connected with the proxy port P. In this configuration, the test request sent through the test browser 1 may be intercepted by the automatic detection tool 3, as in one embodiment, by selecting a proxy scan by FoxyProxy and specifying a proxy IP address. The manual detection tool 2 is an optional existing manual vulnerability detection device, when the manual detection tool 2 needs to be used, a manual detection tool bump configured on a security engineer computer is selected through Foxyproxy, and then a proxy server is arranged in an operation interface provided by the manual detection tool bump. Under the configuration, when the test end browser 1 sends a test request, the manual detection tool bump intercepts the data packet of the test request and provides the data packet to the security engineer through the operation interface, and the security engineer manually tamps the data packet of the test request, manually checks a returned result, and manually judges whether a vulnerability risk exists.
As shown in fig. 2, the item detection management module 4 includes a management interface 41 and a function list generating unit 42, where the management interface 41 is configured to provide data display of setting of security detection parameter items, detection processes, and results. The management interface 41 can set or select a test item name, such as xxx service, and under the service, a plurality of tested service lines can be included, and the service line name is set or selected; setting or selecting a domain name of a test item, such as xxx.yy.com, to limit the domain name related to the test item, and not testing the domain name related to a non-item, so as to avoid extra performance overhead; the IP address of the testing end is set or selected, for example, 10.100.x.x, to limit the IP address of the PC of the security engineer performing vulnerability detection, so that the testing end browser 1 is designated, the source of using the system can be limited, and the system is prevented from being abused. The parameters include parameters for automatic detection, such as request concurrency number, excluded file type, excluded domain name list, various keywords for identifying service function, related configuration of authority authentication mode, related parameters related to test target, such as development framework, development language, corresponding parsing logic, and the like, and various parameters, verification conditions, verification processes, and the like, related to detection strategy during automatic detection. In order to reduce test cases and reduce the test pressure on the test environment, the page fault echoing state is set to be 'yes', so that in the detection process of vulnerabilities such as any file downloading vulnerability and fault-revealing type SQL injection vulnerability, the response returned from a test target can contain fault echoing content, and the detection result can be obtained by using fewer test cases. The management interface 41 further includes a display area for various data, such as a detection result display area, for displaying detection process data and result data of various vulnerabilities, such as obtained vulnerability risk names, risk levels, risk parameters, corresponding function paths, and the like, and includes corresponding detailed information, such as original request header information, scanning information, and the like. The function list generating unit 42 is connected to the management interface 41, and generates a to-be-tested function point list according to the parameter of the test target and the source code of the test target configured by the management interface 41, where the function point list includes a URL (Uniform Resource Locator) list field including URLs of each function point to be tested; the system also comprises a detection state field used for indicating whether the current function point is detected or not; the function point list also includes a URL function identification field, wherein the URL function identification field includes a plurality of function identifications to indicate the service functions of the current function point, such as account authority, information storage, upload function, sensitive function, user private function, and the like. The function point list can be displayed in a corresponding display area of the management interface. The monitoring unit 43 monitors the detection process according to the monitoring index, and generates a corresponding monitoring notification according to the monitoring result to be displayed on the management interface 41. For example, the total number of vulnerabilities, the detected number of vulnerabilities, vulnerabilities at risk, detected functional points, undetected functional points, detailed data in the detection process, etc. are displayed in a graphical manner.
Fig. 3 is a flowchart of a vulnerability detection method according to an embodiment of the present invention, where the method for security engineers to perform vulnerability detection includes the following steps:
in step S1, the security engineer configures or selects the item through the management interface 41 of the item detection management module 4. When the test target is not the first detection, the management interface stores the configuration performed in the last detection, and all the configurations in the last test can be obtained by selecting the corresponding name in the project name parameter item. The safety engineer can directly use the last configuration and can also improve the last configuration according to own experience. Therefore, the invention can share the detection experience and the test strategy among safety engineers, and can improve the test strategy on the basis of the experience of predecessors, thereby further improving the test strategy, further improving the test effect, increasing the reliability of the test and reducing the test period. If the test target is detected for the first time, setting a domain name of the test target, setting an IP address of a test end browser used by a security engineer, setting a port of a proxy server, setting bugs to be detected, and setting a verification type and a corresponding detection strategy for each bug. The detection strategy comprises a verification condition, a verification process, various parameters used in verification and the like. Of course, relevant parameters related to the test target, such as the address where the source code is located, the resolution logic, and the like, are also included.
In step S2, the function list generating unit 42 of the item inspection management module 4 generates a function point list based on the parameters about the test targets and the source codes of the test targets configured by the management interface 41. In one embodiment, the function point list includes a URL list field, a URL function identification field, a function point identity field, a detection status field, and the like. The URL list field records a URL corresponding to the function point, and further comprises a parameter field and a parameter value field. The URL function identification field includes various identification contents for indicating the service function, such as one or more of account authority, information storage, upload function, sensitive function, and user private function, which should be marked as a state of "yes (Y)" or "no (N)". The function point identity field may in one embodiment be a hash value calculated based on the URL and the parameters, which is a unique identification of the identity of the function point. The detection status field may represent by a number that the function point is in an undetected initialization state, in a different state of passing detection, at risk of detection, detected but failed detection, and so on. Specifically, the function list generating unit 42 obtains the name, development language, and framework of the test item according to the configuration of the test item in the management interface 41, selects a corresponding parsing logic, reads the source code of the test item from the source code management library, and parses the source code according to the parsing logic to obtain the function point to be detected, thereby generating the function point list. Wherein, different development languages and frames correspond to different analysis logics. The development language is, for example, PHP, JAVA,. NET, ASP, Ruby, Python, Vue, etc., and the development framework is, for example, an MVC-based open source framework or a custom framework, a three-layer architecture-based custom framework, or some other framework. A procedure for generating a function point list will be briefly described, taking PHP as a development language and MVC-based open source framework as an example:
php is obtained by configuring a Web container.
Php, finding out an application target and a name through regular matching, and outputting the application directory.
And then obtaining a Controller path according to the application directory, and obtaining all file lists of the Controller path.
And then analyzing the files in the file list to obtain the names of the controllers in the files and the names of the methods in the controllers, and removing privatization methods such as private, initial and protected.
And adopting regular expression matching to obtain the user input parameter name. And finally outputting a function list, parameters, a request type and a unique hash value serving as the function point identity. The hash value is calculated based on the URL and the parameter.
In step S3, the security engineer clicks a function point of the test target through the test end browser 1 to send a test request.
In step S4, the test request sent by the test browser 1 is sent to the test target T through the proxy port P, and a response packet returned by the test target T is received.
When the manual detection tool 2 is configured, in step S51, the security engineer intercepts the test request packet through the manual detection tool 2, determines whether there is a risk based on the results returned by manual tampering and manual judgment of the security test item document, and records the detection result, which is referred to as a first detection result, as needed. It should be noted that this step is not necessarily performed for each function point detection, and when the safety engineer doubts about the automatic detection result, the manual detection in this step may be performed, the automatic detection result may be verified by the manual detection result, or the manual detection result may be verified by the automatic detection result. The accuracy of vulnerability detection can be determined through mutual verification, the reliability of an automatic detection result is determined, and the detection level of a safety engineer can be determined through the automatic detection result when the reliability of an automatic detection tool is determined, so that the service capability of the safety engineer can be detected.
Step S52, the automatic detection tool 3 obtains the mirror image data packet of the test request from the test end browser from the proxy port P, performs vulnerability detection according to a preset detection policy, and records a detection result, which is referred to as a second detection result.
In step S6, the safety engineer determines whether there are any functional points that have not been detected, and if the safety engineer determines to end the detection by considering that all the functional points have been detected, a detection end command is issued in step S7, such as clicking an end button on the management interface 41. If the safety engineer considers that there are any function points without detection, the method returns to step S3 to re-send the test request for the new function point.
In step S81, the monitoring unit 43 inquires about the detection completion progress of the function point list when receiving the detection end instruction. For example, look at the detection status field of each function point.
In step S82, it is determined whether all the function points in the function point list have been detected. Wherein, it can be determined that the detection is completed by looking at the detection status field of each function point, if the number in the detection status field represents the initialization status, it indicates that the function point is not detected, then a notification for reminding to continue detection is sent out in step S83 and displayed on the management interface 41; if all the function points in the function point list have been detected, step S91 is executed. When the safety engineer finds that the detection is not ended when the safety engineer presses the end button and a notification requesting the safety engineer to continue the detection occurs, the step S3 is executed to complete the corresponding detection according to the undetected function point in the notification, so that the omission of the detection due to human reasons can be avoided.
In step S91, the monitoring unit 43 monitors the URL function identification field corresponding to each function point. In one embodiment, each function point URL function identification field includes URL identification contents such as account authority, information storage, upload function, sensitive function, and user private function, etc. to indicate the service function of the function point, and the corresponding URL identification contents include two states of "yes" and "no", and each function point is marked as "yes" or "no". Whether the function point completes the identification of the service function can be monitored by inquiring whether each URL identification content of each function point identifies yes or no.
Step S92, determining whether the URL function id field of each function point is marked, if the URL function id field of one or some function points in the function point list is not marked, then a mark reminding notification is sent in step S93, and displayed on the management interface 41. If the URL function identification field of each function point is marked, the detection process is ended. When the security engineer finds that his detection is not finished when he presses the end button and a notification is presented that requires him to mark the URL function identification field, the URL function identification field of the function point is identified in the function point list in step S94, according to the function point indicated in the notification. And then performs step S92. Wherein, in some embodiments, the identification content of the URL function identification field that requires the security engineer to identify includes sensitive functions and account functions. In these embodiments, each function point in the function list is identified by human judgment of the safety engineer.
In other embodiments, upon receiving the end-of-detection command, steps S91-93 may be performed before steps S81-83 are performed. Or the two processes are executed simultaneously, namely the two steps are not in sequence.
The invention can obtain the function point list by analyzing the test target source code, and can finish the detection only when the detection of all the function points in the function point list is finished, thereby realizing the full coverage of the service function points. All configurations are stored in corresponding parameter items after detection is completed each time, research results and experiences of safety engineers can be accumulated and shared with other safety engineers. The invention can collect the historical information of the test items and provide a basis for subsequent safety tests. For a test project, after the function point is subjected to function identification, the identification is stored in the identification field, and after the identification and rechecking of an engineer are carried out during the first test, the engineer identification is not needed any more, so that the flow is simplified for subsequent detection, the dependence on the engineer is reduced, and the system can automatically and accurately execute automatic detection.
Fig. 4 is a schematic block diagram of an automatic detection tool according to an embodiment of the present invention, where the automatic detection tool includes a packet obtaining module 31, a packet analyzing module 32, and a detection module 33, where the packet obtaining module 31 is connected to a proxy port P, obtains a mirror image packet sent by a test browser 1 to a corresponding function point of a test target T from the preset proxy port P, sends an automatic test request packet constructed by the automatic detection tool to the test target T, and receives a response packet returned by the test target T. The data packet analysis module 32 is connected to the data packet acquisition module 31, and is configured to analyze the mirror image data packet and the response data packet, and acquire at least a URL of a removed parameter, a parameter and a parameter value, a function point identity identifier, and a function point detection identifier from the mirror image data packet; and at least acquiring response content from the response data packet. As shown in fig. 5, the data packet analysis module 32 includes a request packet parsing unit 321, a response packet parsing unit 322, a marking unit 323, a hash value calculation unit 324, and a cleaning unit 325.
The request packet parsing unit 321 is configured to parse a request Header (Header), request parameters, and user credentials of a mirror data packet. Parameters in the request Header (Header) such as Cookie, x-forward, refer, user-agent and the like can be obtained by analyzing the request Header of the mirror image data packet. Checking parameter names according to corresponding configuration in a management interface 41, then eliminating parameters which do not need to be detected, then judging whether the remaining parameters have Cookie fields, if so, eliminating the Cookie, obtaining a request header parameter at this moment, and informing a marking unit 323 to mark 'discovery authentication marks' in URL function identification fields of corresponding function points in a function point list; if there is no Cookie parameter in the parameters, the notification marking unit 323 marks "authentication identity not found" in the URL function identification field of the corresponding function point in the function point list. When analyzing request parameters, firstly, according to request types, such as a GET request type or a POST request type, corresponding URL analysis is performed, and GET parameters or POST parameters from the URL analysis. The cleaning unit 325 is connected to the request packet parsing unit 321, and determines whether the parameter value includes a test case, and if so, deletes the test case, thereby obtaining the URL from which the parameter is removed, the parameter, and a corresponding parameter value. When the user credentials are analyzed, the authority authentication mode, such as a Cookie mode or an API mode, configured in the management interface 41 is read, and the authentication identification keyword, such as a sign, token, or key, in the configuration is read. And if the authentication mode in the mirror image data packet is the Cookie mode, analyzing the Cookie field to obtain parameter names such as username and ci _ session. Wherein ci _ session is an authentication credential parameter, which cannot be accessed after being tampered, so that the authority class detection is required, and therefore, the URL function identification field needs to be marked with a "discovery authentication identification", and at this time, the authentication credential is removed, so as to obtain a request parameter with a fixed value removed. If the authentication mode in the mirror image data packet is the API mode, inquiring whether the parameter name matches with the authentication keyword configured in the management interface, if so, removing the authentication keyword, and notifying the marking unit 323 to mark the "found authentication mark" in the URL function identification field of the corresponding function point in the function point list. If the parameter name does not match the authentication keyword configured in the management interface, the notification marking unit 323 marks "authentication not found" in the URL function identification field of the corresponding function point in the function point list. The URL, the parameter and the parameter value of the removed parameter are obtained through the analysis of the request packet analyzing unit 321.
The response packet parsing unit 322 obtains a response status code, such as a code indicating success, redirection, client error, or server error, from the response packet, and obtains information such as the length of the response packet, the content of the response packet, and the like through header parsing.
The marking unit 323 is connected to the request packet parsing unit 321, and after the parsing by the request packet parsing unit 321 is completed, reads the parsed parameters and parameter values, and determines whether the service function of the function point is an upload function or not and whether the service function belongs to a private function according to the configured parameters of the management interface 41. And marking the uploading function and the user private function of the function point URL function identification field as 'yes' or 'no' according to the determined result. Further, when the marking unit 323 marks "find the authentication identifier" according to the notification of the request packet parsing unit 321, a confirmation notification is sent to the management interface 41 to remind the security engineer to identify "account permission" in the URL function identifier field of the function point according to the mark. In addition, in the detection process, the marking unit 323 uses an open-source crawler engine to crawl the function point according to needs, and if the function point can be crawled, the 'account authority' of the function point URL function identification field is marked as 'no'. For example, an A account user may access its own private function URL and a function URL on a public home page; when accessing the private function URL of the a account and the public home function using the NoAuth (unregistered user) account, the result should be that the access to the function URL private to the a account fails and the access to the public home function URL succeeds. Therefore, in the detection process, if it is found that both the account number a and the account number NoAuth can access one function point URL, there is a possibility of an authority risk, in order to check such an authority risk, the detection module starts the crawler engine to use the NoAuth (unregistered user) account or General Auth (General user) to grab the function link, and if the function link can be grabbed, it indicates that the function point does not have the authority function, and the function point may be the aforementioned common home page function, so that the marking unit 323 is notified to mark the "account authority" of the function point URL function identification field as "no".
The hash value calculation unit 324 is connected to the request packet parsing unit 321, and calculates a first hash value according to the URL of the removed parameter and the parameter, and calculates a second hash value according to the URL of the removed parameter, the parameter, and the parameter value. The first hash value is used as the identity of the function point, and the second hash value is used as the detection identity of the function point.
The detection module 33 is connected to the packet analysis module 32, and performs vulnerability detection based on the mirror image packet and its analysis result according to a preset detection strategy. In the invention, a plurality of loopholes to be detected are grouped according to the verification types of the loopholes, and corresponding detection processes are set for loophole groups of different verification types. And the corresponding checking model unit executes the checking process of the corresponding checking type. As shown in fig. 6, the detection module 33 includes a detection management unit 330 and a plurality of detection model units 331, 332, and … … 33n, and the detection management unit 330 controls the plurality of detection model units to sequentially perform vulnerability detection according to a detection flow. For example, according to the detected operating state, the detection management unit 330 divides a plurality of vulnerability groups into an initialization vulnerability group and an operating state vulnerability group, and sets different detection strategies. For example, the initial vulnerability group is detected first, and then the operation state vulnerability group is detected. And the initialized vulnerability group completes detection by one-time request, and for the operation state vulnerability group, a detection strategy combining coarse granularity and fine granularity or a detection strategy combining coarse granularity and result verification is implemented according to the vulnerability type. When coarse-grained detection is carried out, a small number of test cases, such as 1-5, are used, detection is completed through one request, and after suspected risks are obtained, fine-grained detection is carried out or result verification is carried out according to vulnerability types. The detection management unit 330 also maintains a request record table that can be displayed through the management interface 41, in which details of vulnerability detection are recorded, such as original test request data packet, URL, function point identity, function point detection status, risk type, risk level, and so on. Each detection model unit is used for completing the detection of the vulnerability group of one check type. The detection management unit 330 coordinates a plurality of detection model units to detect the vulnerabilities of the verification types thereof according to the scanning strategy, and each detection model unit completes the detection of a vulnerability group of a verification type according to the instruction of the detection management unit and the detection strategy.
As shown in fig. 7, the process of detecting a function point by the detection module 33 includes the following steps:
in step S1a, the detection management unit 330 receives the mirror image packet and the analysis result thereof from the packet analysis module 32.
In step S2a, the detection management unit 330 compares the first hash value with the hash value in the id field in the function point list.
Step S3a, judging whether the first hash value as the function point identity is in the function point list, if not, adding the data such as URL address, parameters and the like in the mirror image data packet into the corresponding fields of the function point list in step S4a, thereby making up the function points which are missed when the function point list is generated; if so, step S5a is performed.
In step S5a, the detection management unit 330 refers to the function point detection identification field in the request record table.
Step S6a, determining whether the second hash value is located in the request record table, that is, whether the second hash value is already included in the function point detection identifier field, and if the second hash value is located in the function point detection identifier field, it indicates that the function point has been detected, at this time, no detection is needed, and the detection of the function point is ended, so that repeated detection of the function point can be avoided. If not, step S7a is performed.
Step S7a, scanning the initialization leak group for initialization detection. The detection management unit 330 controls the detection model units corresponding to the initialized vulnerability group to sequentially perform vulnerability detection according to vulnerability types. In one embodiment, the initialized vulnerability group is an initialized response content verification type, and the corresponding detection model unit 331 sends a test request to the vulnerability types included in the initialized vulnerability group based on the mirror image data packet (i.e., the original data packet) respectively, and then completes the detection. The corresponding vulnerability types are server fingerprint leakage vulnerability, HTTP request mode vulnerability and sensitive information plaintext transmission vulnerability.
Taking the example of detecting a server fingerprint leakage vulnerability as an example, the flow of the detection model unit 331 is described, as shown in fig. 8:
in step S1b, the inspection model unit 331 puts the mirror image packet in the request queue. In the actual detection process, the mirror image data packets of multiple function points may need to perform initial response content check type vulnerability detection, so the mirror image data packets detected by the function points are placed in a request queue for queuing.
Step S2b, determining whether the mirror image data packet reaches the top of the request queue, and if so, sending the mirror image data packet to the test target T through the data packet obtaining module 31 in step S3 b; if not, wait.
Step S4b, the response content of the response packet obtained after the analysis by the received packet analysis module 32.
Step S5b, the response content is analyzed to determine if there is a risk. Corresponding to server fingerprint leakage loopholes, whether parameter values of response content contain numbers or not is identified through a regular expression, and when the parameter values of the response content contain the numbers, risks are determined to exist, for example: when PHP 5.2.5 is included in the response, the specific version of the number "5.2.5" is included, confirming that there is a risk; when no number is contained in the response contents, no risk is confirmed, for example, when the response contents are "X-Power-By: PHP".
Step S6b, records the detection result in the request record table.
And step S8a, scanning the operation state vulnerability groups to sequentially detect vulnerabilities. The detection management unit 330 controls the plurality of detection model units to sequentially complete the detection of the corresponding vulnerabilities. One vulnerability detection flow of the run-state vulnerability group is shown in fig. 9:
step S1c, constructing a first automatic test data packet based on a vulnerability verification type, wherein the first automatic test data packet is an original mirror image data packet for vulnerabilities of a response content verification type and a request replay verification type; for the loophole of the request response verification type, a first automatic test data packet is formed by adding a preset test case in the parameter value of the original mirror image data packet; for the bug of the replacement request response verification type, a first automatic test data packet is formed by replacing the parameter value of the original mirror image data packet with a preset test case; for the loophole of the login certificate replacement check type, a first automatic test data packet is formed by replacing the original user authentication certificate in the original mirror image data packet with a preset authentication certificate.
Step S2c, the corresponding inspection model unit puts the first automatic test packet into its internal request queue.
Step S3c, determining whether the first automatic test packet reaches the top of the request queue, and if so, sending the first automatic test packet to the test target T through the packet obtaining module 31 in step S4 c; if not, wait.
Step S5c, the response content of the response packet obtained after the analysis by the received packet analysis module 32.
Step S6c, the response content is analyzed. And when analyzing the response content, analyzing according to the verification type of the vulnerability and the corresponding strategy. For example, for vulnerabilities such as a request response verification type, a replacement request response verification type, a response content verification type and the like, whether the response content contains content preset in the detection strategy is analyzed; for the login certificate replacement check type, comparing the contents of the two responses; for the request replay check type, a response to the original request and a response to the replay request are compared.
Step S7c, judging whether the requirement of suspected risk is satisfied, if not, indicating no risk, confirming the detection is passed in step S8c, and recording the detection result into the request record table; if so, indicating that there is a possible risk, execution is step S9 c. For example, for a reflective cross-site scripting risk belonging to a request response verification type, when the response content shows back the original form, the suspected risk requirement is considered to be met, and the suspected risk is determined; for the unauthorized access loophole belonging to the login certificate replacement verification type, when the contents of the two responses are highly similar, the suspected risk requirement is considered to be met, and the suspected risk is confirmed; for cross-site request forgery bugs belonging to a request replay check type, when two request responses are completely consistent, the suspected risk requirement is considered to be met, and the possibility of risk is confirmed.
And step S9c, acquiring a secondary test condition of the vulnerability with the suspected risk. The secondary test conditions are different according to different vulnerability types and are recorded in the detection strategy, and the secondary test conditions can be obtained by inquiring the detection strategy. For example, some have no limitation of test conditions, such as a reflection-type cross-site scripting risk belonging to a request response verification type, a sensitive information leakage vulnerability belonging to a response verification test, and the like; some test conditions provide for verification based on the status of the relevant tag content in the URL tag field.
And step S10c, performing secondary test or verification according to the corresponding test conditions.
Step S11c, judging whether the structure of the secondary test or check is risk, if yes, modifying the risk state in the request record table from 'suspected' to 'confirmed' in step S12c, and ending the detection; if the risk is not detected after the secondary test or verification, step S13c modifies the risk status in the request record table from "suspected" to "false positive" and ends the detection.
In step S10c, the flow and judgment criteria of the secondary test or verification are different for different test conditions. For example, for a reflective cross-site scripting risk belonging to a request response verification type and a sensitive information leakage vulnerability belonging to a response verification test, no test condition is needed, and then a secondary test is directly performed, for example, a test request data packet is reconstructed and sent to a test target, and returned response content is analyzed. And according to whether the requirement of the risk is met, modifying the suspected risk state in the request record table into confirmation or false report.
For a vulnerability which needs to be checked against the mark state of the mark content in the URL function identification field, for example, when a suspected risk is found by performing a coarse detection on an unauthorized access vulnerability, whether the mark of the mark content of the account right in the URL function identification field of the function point is "yes" (Y) or "no" (N) is inquired, if the mark of the "account right" is "yes" (Y), it is determined that the risk of the vulnerability type really exists, the risk state in the request record table is modified from "suspected" to "confirmed", and if the mark of the "account right" is "no" (N), the risk state is modified from "suspected" to "false alarm". For another example, when a suspected risk occurs in the detection of the horizontal override hole, firstly, whether the mark of the identification content "account authority" in the URL function identification field of the function point is "yes (Y)" or "no (N)" is queried, and if the mark of "account authority" is "no (N)", the risk state is modified from "suspected" to "false alarm". If the mark of the account authority is 'Y', then inquiring whether the mark of the user private function is 'Y' or 'N', if the mark is 'N', then modifying the risk state from 'suspected' to 'false positive'. If yes, then confirm that the risk of the vulnerability type does exist, and modify the risk status in the request record table from "suspected" to "confirmed".
When the parsing function point determines that the function point has the user authentication credentials and the "account authority" of the function point is not marked by the security engineer, the crawler engine is started, the NoAuth (unregistered user) account or General Auth (General user) is used for grabbing the function link, if the function link can be grabbed, the function point does not have the authority function, the function point may be the aforementioned common home page function, and therefore the marking unit 323 is notified to mark the "account authority" of the function point URL function identification field as "no". According to the aforementioned verification process, the suspected risk that the "account authority" is marked as "no" is eliminated.
The checking time may occur during the detection process, or may be before the detection process is completed and the test is finished. For example, for the verification of the "information storage function", after the detection is completed and the security engineer marks all URL functions, if the "information storage function" of the function point is marked as "yes", the security engineer needs to manually write a specified payload in the form item: sectxs' "< >. The system automatically replays all the test requests with the information storage function marked as N and checks the response packet of each test request.
In one embodiment, as shown in FIG. 10, one inspection model unit 331 includes a request queue 3311, a sending subunit 3312, a receiving subunit 3313, and a checking subunit 3314. The request queue 3311 is configured to store a test request data packet sent to a test target, where the test request data packet is a mirror image data packet used when detecting a vulnerability in an initialized vulnerability group, and may also be a test request data packet constructed when performing coarse-grained detection on a vulnerability in a running vulnerability or a new test request data packet used for fine-grained detection. The sending subunit 3312 takes out the test request data packets from the request queue in sequence, and sends the test request data packets to the test target through the agent port via the data packet obtaining module 31. The receiving subunit 3313 is connected to the packet analysis module 32, and receives the response content analyzed by the packet analysis module with respect to the response packet returned from the test target. The checking subunit 3314 is provided with a detection policy corresponding to the type of checking it detects, checks the returned response data according to the detection policy to determine whether there is a risk of a corresponding bug, and sends the checking result to the detection management unit 330. For example, for a reflective cross-site scripting risk vulnerability of a request response verification type, 4 test cases are included in the transmitted test request data packet, and the verification subunit 3314 verifies whether the response content is echoed back as it is, and if so, determines that the response content is a suspected risk.
Among some inspection model units, such as an inspection model unit for performing vulnerability group inspection of request response verification type, replacement request response verification type, and login credential replacement verification type, a test request constructing subunit 3310 is further included, which constructs a new test request packet according to an inspection policy. For example, in the detection model unit for detecting the request response verification-type vulnerability group, the test request construction subunit 3310 adds test cases to the parameter values of the mirror image data packet, and combines the added test cases and the original parameter values into an attack load to form the test request data packet. The number of test cases may vary depending on the vulnerability type, coarse (level 1) or fine (level 2). For example, for any file download bug belonging to a request response verification type, 1 test case is used in coarse-grained detection, and more than 1 test case is used in fine-grained detection. For the reflection-type cross-site script risk belonging to the request response verification type, 4 test cases are used in coarse-grained detection, and more than 4 test cases are used in fine-grained detection. For any file upload vulnerability belonging to a replacement request response verification type, 3 test cases are used in coarse grain detection.
In the detection model unit for detecting the replacement request response verification-type vulnerability group, the test request construction subunit 3310 replaces the parameter values in the mirror image data packet with test cases, thereby forming a test request data packet. When the verification type of the leaky port group is a login credential replacement verification type, 3310 replaces the user authentication credentials in the mirror image data packet with authentication credentials of users of different levels, respectively, to form a plurality of automatic test request data packets. For example, for an unauthorized access vulnerability belonging to a login credential replacement verification type, the user authentication credentials in the mirror image data packet are replaced by the authentication credentials of a high-level authority user such as an unregistered user account credential and an administrator, respectively, to obtain two test request data packets. Correspondingly, the checking subunit 3314 compares the response contents returned by the two test requests, and if the response contents are highly similar, it determines that the two test requests are suspected to be bugs.
The detection management unit 330 receives the verification result obtained by the detection model unit 331, and determines whether to perform fine-grained detection or result verification according to the vulnerability type when the detection management unit 330 receives that the verification result sent by the detection model unit is suspected risk. For example, for a request response verification type leak group, such as a reflection-type cross-site scripting risk, an error-display-type SQL injection leak, an arbitrary file download leak, and the like, when a coarse-grained detection result is a suspected risk, a certain amount of test cases are reused to construct a test request data packet, fine-grained detection is performed, and if the coarse-grained detection result is still a risk according to a detection strategy, the current suspected risk can be confirmed to have the leak. For example, for suspected risks of any file download vulnerability and error-prone SQL injection vulnerability, a new test case is used for verification in the fine-grained (level 2) detection process, and the vulnerability is confirmed if the contents of two requests and responses are different. For another example, in the case of a suspected risk of a reflection-type cross-site scripting risk, if a new test case is used in the fine-grained (level 2) detection process, the returned response content is returned and displayed as it is, and then a vulnerability is confirmed. When the suspected risk is detected as any file upload vulnerability, the detection management unit 330 directly determines the suspected risk as a risk.
For suspected risks of some vulnerabilities, the coarse-grained detection result needs to be checked according to the service function of the function point. Thus, in one embodiment, the detection module 33 further comprises a result checking unit 334, connected to the detection management unit 330, configured to receive a result checking instruction from the detection management unit 330, and check the suspected risk according to a checking policy. The vulnerabilities to be checked are, for example, vulnerabilities related to an authority class, such as a horizontal override vulnerability, an unauthorized access vulnerability and a vertical override vulnerability, and also vulnerabilities related to sensitive functions, such as a cross-site request forgery vulnerability. Therefore, the result checking unit 334 may be one unit, and check the vulnerabilities requiring result checking one by one according to a corresponding checking mechanism. The result verification unit 334 may be a plurality of units, and each unit corresponds to different bug result verifications.
For permission type vulnerabilities such as a horizontal override vulnerability, an unauthorized access vulnerability and a vertical override vulnerability, the result verification unit 334 queries the identification content in the URL mark field as a mark of the account permission, and starts the crawler engine to capture the functional point link when the account permission is not marked. For example, for an unauthorized access vulnerability, starting a crawler engine to capture the function point link using an unregistered account; and for the vertical unauthorized vulnerability, starting a crawler engine to capture the function point link by using a common account. When the function point link is grabbed, the account authority in the function point URL mark field is marked as 'no', and the risk is eliminated, namely the current risk state is modified from 'suspected' to 'false positive'.
For the horizontal override vulnerability and the vertical override vulnerability, when the result verification unit 334 queries that the mark responding to the account authority in the URL mark field is "yes", the mark of the user private function is queried; and when the mark of the user private function is 'yes', confirming the risk, namely, modifying the current risk state from 'suspected' to 'confirmed', and eliminating the risk when the mark of the user private function is 'no'.
For unauthorized access vulnerabilities, the risk is excluded when the result verification unit 334 queries a flag "yes" in response to the account permissions in the URL flag field.
In addition, for the cross-site request forgery vulnerability, the result verification unit 334 queries the sensitive function of the URL mark field of the current function point, and when the function is marked as "yes", confirms the risk; when it is marked "no", the risk is excluded.
The detection management unit 330 detects all functional points and determines whether all the identification contents in the URL marking fields are completely marked, queries the identification contents in the URL marking fields of each functional point as a mark for information storage, and performs detection of the storage-type cross-site script vulnerability again for the functional points marked as "no". The method comprises the steps that a replay request instruction is sent to a detection model unit for detecting the storage type cross-site scripting vulnerability, the detection model unit for detecting the storage type cross-site scripting vulnerability sends out a test request data packet again according to the instruction, a check subunit of the detection model unit compares the response content of the current request with the response content during rough detection, and whether the risk exists or not is confirmed according to the comparison result.
The vulnerability is divided into different groups according to the verification type, such as a request response verification type, a replacement request response verification type, a login credential replacement verification type, a response content verification type or a request replay verification type. And the loopholes of different verification types have different detection strategies aiming at the loophole types and are respectively executed by one detection model unit. Because the types of the vulnerabilities are continuously changed along with the version updating of the test target and other reasons, new vulnerabilities can appear at any time, and when the new vulnerabilities appear, the method only needs to match the corresponding check types for the new vulnerabilities and add the new vulnerabilities to the corresponding groups because of the adoption of the detection model unit mode. Or, when there is no existing verification type matched with the verification type, one detection model unit can be added to the verification type, and the operation of other detection model units is not influenced. The modularized detection mode has good expansibility, and certain vulnerability detection can be added or deleted at any time. Because the detection flow of the detection model unit is standardized, the detection items are also fixed in a standardized form, and when vulnerability detection is added, a security engineer only needs to modify related parameter items at the front end (such as a management interface provided by the invention), such as adding vulnerability names and types, setting corresponding verification types, or adding new verification types, and the like, so that the dependence on developers is greatly reduced, and the intervention and workload of the developers are reduced. And because of the standardization of the detection process and the strategy, a primary security engineer only with basic knowledge can complete final vulnerability review, and a high-level engineer concentrates on detection strategy formulation and new rule research. The invention divides the detection flow into two steps of initial coarse-grained safety detection and rechecked fine-grained safety detection, the coarse-grained safety detection is only configured with a small amount of optimal test cases, for example, a large amount of accurate test cases are called for rechecking after suspected bugs appear, so the requirement on the performance of the test server is not high, the security detection of the bugs can be completed quickly under the test environment with poor performance, the accuracy is high, and the efficiency is high.
In the existing vulnerability detection method and tool, a security engineer usually completes vulnerability detection on a plurality of function points of a test target. In an embodiment of the present invention, in the process of executing the vulnerability detection method to detect the test target, the vulnerability detection system needs a security engineer to send a test request for the test target function point through the test end browser according to the detection flow. This creates a need to determine whether the safety engineer has performed a full number of tests on all functional points according to the test flow. Fig. 11 is a flowchart of a vulnerability detection flow verification method according to an embodiment of the present invention, which verifies a detection flow of a security engineer when the security engineer performs vulnerability detection to determine whether the security engineer completes vulnerability detection of all function points according to the detection flow. The authentication method comprises the following steps:
step S100, obtaining configuration parameters of a test item, wherein the configuration parameters at least comprise a test target parameter group, a proxy server port, a vulnerability and a detection strategy thereof. The test target parameter group is various parameters corresponding to the test target, such as a domain name, a development framework, a development language, corresponding resolution logic, a source code storage address and the like.
Step S101, generating a function point list based on a source code in a test item parameter group, wherein the function point list at least comprises a URL (Uniform resource locator) of a function point, a parameter and a function point identity.
Step S102, when a safety engineer sends a test request for a test project function point through a test end browser according to a detection process, a mirror image data packet of the test request is intercepted.
Step S103, detecting the vulnerability of the function point based on the mirror image data packet according to a preset detection strategy, and recording the detection result of the function point to generate a detection result list. The detection process is as described above and will not be described herein. The detection result list comprises the request record table, wherein the detected function point URL, the function point identity, the function point detection identity, whether the function point is risky or not and the like are recorded.
Step S104, judging whether a detection ending request is received. When the security engineer considers that the detection is finished, he clicks the end button through the management interface 41 to finish the current vulnerability detection process. When receiving the detection end request, executing step S105, if not, returning to step S102 to continue intercepting the mirror image data packet of the test request for a certain function point.
Step S105, comparing the detection result list with the function point list. For example, compare the function point identities in the two lists.
Step S106, judging whether the detected function point covers the function point in the function point list. If all the function points in the function point list have been detected, it is confirmed in step S108 that all the function completion detection is completed, and the verification process of the present embodiment is ended. If there are any undetected function points in the function point list, the detection flow is prohibited from ending in step S107, and preferably, a prompt message for continuing the detection is generated and displayed in this embodiment. Because the detection process cannot be finished and the reminding message exists in the display interface, after the safety engineer checks the message, the safety engineer continuously sends a test request to the function point in the reminding message to perform vulnerability detection. Therefore, the process returns to step S102.
Through the vulnerability detection flow detection method provided by the embodiment, whether a security engineer completes detection of all function points according to the required flow can be detected, and if a missing function point is found, the security engineer can be prompted to continue detection until all function points are detected, so that it can be ensured that no missing function point exists after vulnerability detection is finished.
Fig. 12 is a flowchart of a vulnerability detection process verification method according to another embodiment of the present invention, where steps S200 to S204 in this embodiment are the same as steps S100 to S104 in fig. 11, and are not described again here. When the detection end request is received, step S205 is executed.
Step S205, querying a marking condition of the URL function identification field of the function point in the function point list. And the URL function identification field of each function point in the function point list comprises a plurality of identification contents, and in the mirror image data analysis process in the detection process, the function of the function point is judged according to the analyzed parameters and is marked, or the result crawled by a crawler engine is used for marking in the result verification process. In one embodiment, each identifying content may be marked as yes (Y) or no (N), and is null when not marked. Therefore, whether the marked content is marked or not can be determined by inquiring the current marking state of each marked content of each function point. If the URL mark content of the functional point is not marked, the detection process is prohibited from ending in step S207, and a mark reminding message is generated and displayed on the interface. When the safety engineer receives the reminding message after clicking the end button, the safety engineer can find the function point according to the function point identity in the message and mark the unmarked content of the function point. If all the URL mark contents of all the function points are found to be marked, it is confirmed in step S208 that all the function point URL function point mark contents are marked, thereby ending the verification process of this embodiment.
In the vulnerability detection strategy provided by the invention, for some vulnerabilities, after the suspected risk is obtained through coarse-grained detection, verification is required to be carried out according to the specific service function of the function point so as to confirm or eliminate the suspected risk, so that the service function of each function point needs to be determined. The result verification can be ensured after the identification content is marked, so that the accuracy of corresponding vulnerability detection is ensured.
Fig. 13 is a flowchart of a vulnerability detection flow verification method according to another embodiment of the present invention, where the vulnerability detection system provided by the present invention needs to perform corresponding configuration when starting to detect a test item, and as described above, selects or creates a new test item name, and selects various parameters of the test item, including a vulnerability to be detected, a vulnerability type, and a corresponding detection policy. In one embodiment, default parameters are set for each test item in the system, and security engineers may select or modify the test items as needed, such as adding new vulnerabilities and detection strategies, and modifying current detection strategies. In order to avoid that security engineers set mistakes to cause the corresponding detected bugs to be missed, the method includes the following steps:
step S301, a plurality of first vulnerabilities in the detection result list are obtained to form a first vulnerability list. When the automatic detection tool detects the vulnerabilities of each function point, the vulnerabilities are detected one by one according to the configured vulnerabilities, so that all vulnerabilities obtained in the vulnerability detection result of one function point can be counted. In order to distinguish the bug in the default parameter, the bug obtained in the detection result is referred to as a first bug, and the bug in the default parameter is referred to as a second bug. A first vulnerability list can be obtained by counting vulnerabilities in vulnerability detection results of one functional point.
Step S302, reading the bugs in the configuration parameters to obtain a second bug list.
Step S303, comparing the first vulnerability list with the second vulnerability list.
Step S304, it is determined whether the first vulnerability list covers the second vulnerability list, that is, whether the second vulnerability in the second vulnerability list is the same as part or all of the first vulnerabilities in the first vulnerability list, and if one or more vulnerabilities that are not present in the first vulnerability list exist in the second vulnerability list, which is referred to herein as a third vulnerability, in step S305, the detection flow is prohibited from being ended, and a vulnerability missed detection prompting message is generated and displayed, otherwise, in step S306, it is determined that no vulnerability is missed.
Through the verification process shown in the embodiment, the situation of vulnerability missed detection caused by improper parameter configuration can be prevented.
Fig. 14 is a schematic block diagram of a vulnerability detection flow verification system according to an embodiment of the present invention, which partially overlaps with the vulnerability detection system, and includes a test-side browser 1, an automatic detection tool 3, an item detection management module 4, and a flow verification module 5. The project detection management module 4 obtains configuration parameters of a test project, wherein the configuration parameters at least comprise a test target parameter group, a proxy server port, a vulnerability and a detection strategy thereof; and generating a function point list based on the source code in the test item parameter group, wherein the function point list at least comprises the URL, the parameters and the function point identity of the function point. The test end browser 1 sends a functional point test request data packet to a test target through the proxy port, and receives a response data packet returned from the test target. The automatic detection tool 3 acquires a test request mirror image data packet from a test end browser sent by the agent port, performs vulnerability detection on the function points according to a preset detection strategy based on the mirror image data packet, and records the detection results of the function points to generate a detection result list. The process inspection module 5 is respectively connected with the project detection management module and the automatic detection tool in a configuration mode and is configured to compare the detection result list with the function point list when a detection process end request is sent; and in response to the detected function point in the detection result list not covering the function point in the function point list, prohibiting the detection process from ending.
In one embodiment, as shown in fig. 15, the flow verification module 5 includes one or more of the following verification units: a function point checking unit 51, a URL mark checking unit 52, a vulnerability checking unit 53 and a message generating unit 54, wherein the function point checking unit 51 is used for comparing whether the function points in the detection result list cover the function points in the function point list; when it is confirmed through comparison that the function point in the detection result list does not cover the function point in the function point list, that is, there is no function point detected in the function point list, the function point list is sent to the message generating unit 54, and the message generating unit 54 generates a function point continuous detection reminding message and displays the message on the management interface 41. The URL mark checking unit 52 is configured to query a mark condition of a URL function identification field of a function point in the function point list, and send a notification to the message generating unit 54 when an unmarked identification content is found in the URL function identification field of the function point, where the message generating unit 54 generates a mark reminding message and displays the mark reminding message on the management interface 41. The vulnerability testing unit 53 is configured to compare vulnerabilities in the detection result list with vulnerabilities in the default parameters, and send a notification to the message generating unit 54 when a vulnerability in the detection result list is less than a vulnerability in the default parameters, and the message generating unit 54 generates a vulnerability missing detection reminding message and displays the vulnerability detection reminding message on the management interface 41.
The above embodiments are provided only for illustrating the present invention and not for limiting the present invention, and those skilled in the art can make various changes and modifications without departing from the scope of the present invention, and therefore, all equivalent technical solutions should fall within the scope of the present invention.