CN110233774B - A detection method, distributed detection method and system for a Socks proxy server - Google Patents

Abstract

The invention discloses a detection method, a distributed detection method and a system for a Socks proxy server, belonging to the field of network resource space detection. The GET request message based on HTTP protocol is sent to the host to be tested through the preset ports, and the response information of each port is obtained; the response information of each port is parsed, if it does not contain the first feature string and contains the second feature string , it is determined that the host has opened the Socks proxy service through the corresponding port; otherwise, it is determined that the host has not opened the Socks proxy service through the corresponding port; if the host has opened the Socks proxy service through any port, the host to be tested is identified as the Socks proxy server; otherwise, Identify the host to be tested as a non-Socks proxy server; the detection ends. The invention can effectively solve the problem of low recognition rate of the existing Socks proxy server detection method, and speed up the detection speed through the distributed detection system.

Description

A detection method, distributed detection method and system for a Socks proxy server

technical field

The invention belongs to the field of network resource space detection, and more particularly, relates to a distributed detection method and system of a Socks proxy server.

Background technique

With the rapid development of the network society, various industries are currently relying on the Internet to handle related businesses, and our daily activities such as chatting, shopping, entertainment, and learning are closely integrated with the Internet, and the Internet is closely related to life. Although the Internet has facilitated our lives, the security problems brought by it cannot be ignored.

The Internet is a highly open field. At present, enterprises and users can build their own proxy servers. Users can obtain resources that are difficult to access under normal circumstances through the proxy server. In addition, the proxy server generally has a large buffer, which can speed up access to the network. The speed of resources can also hide your private information. Based on the above purposes, proxy servers such as HTTP proxy, Telnet proxy, PPTP proxy, L2TP proxy, and Socks proxy have been widely used in the Internet. While the proxy server brings convenience to Internet users, it also has certain security risks. Many network attacks and illegal information transfer are realized through illegal agents. For example, hackers use proxy servers to conduct DDoS attacks to hide their information, and criminals use proxy servers to conduct gambling and fraudulent information transfer. Therefore, studying the identification method of proxy server is not only of great significance for further use, but also can provide technical guarantee for network security. Among many proxy servers, Socks proxy server is currently the most widely used proxy server, and it is relatively difficult to detect. Therefore, it is of great significance to study the Socks proxy detection method in production and life.

At present, mainstream detection tools, such as Zmap, ProxyBroker, etc., can detect the Socks proxy server. However, these detection tools are based on the Socks protocol request and response packet analysis method, and can only identify the Socks proxy server without an account and password. For the Socks proxy server with the account and password, during the actual detection, the probe data packets sent are often directly discarded by the host to be tested, and the response information cannot be obtained, resulting in the inability to effectively identify the Socks proxy server. In general, the recognition rate of existing Socks proxy server detection methods is low.

SUMMARY OF THE INVENTION

Aiming at the defects and improvement requirements of the prior art, the present invention provides a detection method, a distributed detection method and a system for a Socks proxy server, aiming at solving the problem of low recognition rate of the existing Socks proxy server detection method.

In order to achieve the above object, according to the first aspect of the present invention, a detection method for a Socks proxy server is provided, which is used to identify whether a host to be tested whose ip is known is a Socks proxy server, including:

(1) Judge whether the host to be tested is open for service, if so, go to step (2); otherwise, go to step (6);

(2) Preset multiple ports for establishing communication connections on the host to be tested;

(3) send the pre-constructed GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port;

(4) Parse the response information of each port. If it does not contain the first characteristic string and contains the second characteristic string, it is determined that the host to be tested has opened the Socks proxy service through the corresponding port; otherwise, it is determined that the host to be tested has not opened the Socks proxy service. Open the Socks proxy service through the corresponding port;

(5) If the host to be tested has opened the Socks proxy service through any port, identify the host to be tested as a Socks proxy server; otherwise, identify the host to be tested as a non-Socks proxy server;

(6) The detection is over;

The first feature string and the second feature string are both pre-collected according to proxy features, the first feature string is used to exclude interference from other proxy servers, and the second feature string is used to identify the Socks proxy server.

The present invention detects the Socks proxy server based on the GET request message of the HTTP protocol, and identifies whether the host to be tested is a Socks proxy server according to whether the response information of the port of the host to be tested contains a characteristic string. A Socks proxy server or a Socks proxy server without an account and password will generate response information corresponding to the GET request message of the HTTP protocol. Therefore, the present invention can identify both a Socks proxy server without an account and password, and a Socks proxy server with an account and password. The Socks proxy server effectively solves the problem of low recognition rate of the existing Socks proxy server detection method.

Further, the first characteristic string is 'SSH', 'FTP', '\xff\xfd\x18\xff\xfd\xff\xfd#\xff\xfd', 'SMTP' or 'HTTP'.

Further, the second characteristic string is '\x05\x00', '\x00[\x00\x00\x00\x00\x00\x00' or 'www.herokucdn.com'.

The above-mentioned first characteristic string is a unique character string in the response information generated by other proxy servers such as SSH, FTP, SMTP, and HTTP for the GET request message of the HTTP protocol; the above-mentioned second characteristic string is the Socks proxy server for the HTTP protocol. The unique character string in the response information generated by the GET request message; because other proxy servers such as SSH, FTP, SMTP, HTTP, etc. will interfere with the detection of the Socks proxy server, the present invention only does not include the above-mentioned first characteristic character string. And the host to be tested including the second characteristic character string is identified as a Socks proxy server, in the process of identifying the Socks proxy server, the interference of other proxy servers is avoided, so the identification accuracy of the present invention is high.

Further, the detection method of the Socks proxy server provided by the present invention also includes: if the host to be tested is identified as the Socks proxy server, then the spatial position of the host to be tested is obtained according to its ip, thereby realizing the physical location of the Socks proxy server;

The invention further realizes the positioning of the Socks proxy server according to the IP, and provides convenience for the supervision of the Socks proxy server.

According to the second aspect of the present invention, a distributed detection method of a Socks proxy server is provided, which is used to identify whether a large-scale IP known host to be tested is a Socks proxy server, including:

Divide all the hosts to be tested into multiple sets of hosts to be tested;

For each set of hosts to be tested, the detection method of the Socks proxy server provided by the first aspect of the present invention is used to detect the hosts to be tested in turn, so as to identify the Socks proxy server therein; the detection of all sets of hosts to be tested is performed in parallel implement.

The distributed detection method of the above-mentioned Socks proxy server divides multiple hosts to be tested into sets of hosts to be tested, so that the detection of the hosts to be tested in the set is executed serially, and the detection of multiple sets is executed in parallel, which can effectively improve the performance. detection rate.

According to a third aspect of the present invention, a detection system for a Socks proxy server is provided, which is used to identify whether a host to be tested whose ip is known is a Socks proxy server, including: a pre-judgment module, a port setting module, a transceiver module, Parsing module and identification module;

The pre-judgment module is used to judge whether the host to be tested has open services, and end the detection when the host to be tested has not opened services;

The port setting module is used to preset a plurality of ports for establishing a communication connection on the host to be tested when the pre-judgment module determines that the host to be tested is open for service;

The transceiver module is used to send the pre-configured GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port;

The parsing module is used to parse the response information of each port, and when the response information does not contain the first feature string and contains the second feature string, it is determined that the host to be tested has opened the Socks proxy service through the corresponding port; and In other cases, it is determined that the host to be tested has not opened the Socks proxy service through the corresponding port;

The identification module is used to identify the host under test as a Socks proxy server when the parsing module determines that the host under test has opened the Socks proxy service through any port; and in other cases, identify the host under test as a non-Socks proxy server;

The first feature string and the second feature string are both pre-collected according to proxy features, the first feature string is used to exclude interference from other proxy servers, and the second feature string is used to identify the Socks proxy server.

According to the fourth aspect of the present invention, a distributed detection system of a Socks proxy server is provided, which is used to identify whether a large-scale IP-known host to be tested is a Socks proxy server, including: a database, a central scheduling node and a plurality of parallel The task execution node of ;

The central scheduling node is used to divide all the hosts to be tested into multiple sets of hosts to be tested, and distribute the detection tasks of each set of hosts to be tested to multiple different task execution nodes, so that the detection tasks of each set of hosts to be tested are execute in parallel;

The task execution node is used for the received host set to be tested, and the detection method of the Socks proxy server provided by the first aspect of the present invention is used to detect the hosts to be tested in turn, so as to identify the Socks proxy server. , and return the task execution result to the central scheduling result, and store the detection result persistently in the database; the execution result is used to display the success or failure of the task execution, and the detection result includes the IP address, port, and spatial location information of the Socks proxy server;

The central scheduling node is further configured to receive the execution results returned by each task execution node, and transfer the failed task from the original task execution node to other task execution nodes to continue execution according to the execution results.

In the distributed detection system of the above Socks proxy server, the central scheduling node and the task execution node constitute a Master-Slave (master-slave) model. After scheduling, each task execution node executes the detection task of the Socks proxy server in parallel. This can effectively improve the detection rate of large-scale hosts to be tested.

Further, the central scheduling node includes: a task distribution module and a failover module;

The task distribution module is used to divide all the hosts to be tested into multiple sets of hosts to be tested, and distribute the detection tasks of each set of hosts to be tested to multiple different task execution nodes, so that the detection tasks of each set of hosts to be tested are execute in parallel;

The failover module is used for receiving the execution result returned by each task execution node, and according to the execution result, the failed task is transferred from the original task execution node to other task execution nodes to continue execution.

The above distributed detection system implements a failover mechanism by transferring the failed detection task from the original task execution node to another task execution node, thereby ensuring that the detection of each host to be tested is normally executed.

Further, the task execution node includes: a prediction module, a port setting module, a transceiver module, a parsing module, an identification module, a storage module, and a return module;

The pre-judgment module is used to judge whether the host to be tested has opened services, and when the host to be tested has not opened services, end the detection of the host to be tested;

The port setting module is used to preset a plurality of ports for establishing a communication connection on the host to be tested when the pre-judgment module determines that the host to be tested is open for service;

The transceiver module is used to send the pre-configured GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port;

The parsing module is used to parse the response information of each port, and when the response information does not contain the first feature string and contains the second feature string, it is determined that the host to be tested has opened the Socks proxy service through the corresponding port; and In other cases, it is determined that the host to be tested has not opened the Socks proxy service through the corresponding port;

The identification module is used to identify the host under test as a Socks proxy server when the parsing module determines that the host under test has opened the Socks proxy service through any port; and in other cases, identify the host under test as a non-Socks proxy server;

The storage module is used to persistently store the detection results in the database;

The return module is used to return the execution result to the central scheduling node;

The first feature string and the second feature string are both pre-collected according to proxy features, the first feature string is used to exclude interference from other proxy servers, and the second feature string is used to identify the Socks proxy server.

In general, through the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:

(1) The detection method of the Socks proxy server provided by the present invention and the detection system of the Socks proxy server, utilize the GET request message based on the HTTP protocol to detect the Socks proxy server, and according to whether the response information of the host port to be tested contains A specific characteristic string to identify whether the host to be tested is a Socks proxy server, because whether it is a Socks proxy server with an account password or a Socks proxy server without an account password, it will generate response information for the GET request message of the HTTP protocol accordingly. Therefore, the present invention can identify both the Socks proxy server without account and password, and the Socks proxy server with account and password, thereby effectively solving the problem of low recognition rate of the existing Socks proxy server detection method.

(2) The detection method of the Socks proxy server and the detection system of the Socks proxy server provided by the present invention accurately set the first feature string for eliminating interference and the second feature string for identifying the Socks proxy server, And the host to be tested that does not contain the first feature string and contains the second feature string is identified as a Socks proxy server, so that the present invention can avoid the interference of other proxy servers in the process of identifying the Socks proxy server. Therefore, The identification accuracy of the present invention is high.

(3) The detection method of the Socks proxy server and the detection system of the Socks proxy server provided by the present invention can also perform spatial positioning of the Socks proxy server according to ip, which provides convenience for the supervision of the Socks proxy server.

(4) The distributed detection method of the Socks proxy server provided by the present invention and the distributed detection system of the Socks proxy server are based on the detection method of the Socks proxy server provided by the present invention, and are executed in a distributed manner in multiple tasks Nodes perform Socks proxy server detection at the same time, so that tasks can be executed in parallel. Therefore, the present invention can effectively improve the detection rate on the basis of ensuring a higher recognition rate.

(5) In the distributed detection system of the Socks proxy server provided by the present invention, when the central scheduling node fails to detect any host to be tested, it will transfer the detection task of the host to be tested from the original task execution node to the Another task execution node is used to restart the detection task of the host to be tested, thereby realizing a failover mechanism to ensure that the detection of each host to be tested is normally performed.

Description of drawings

1 is a flowchart of a detection method for a Socks proxy server provided by an embodiment of the present invention;

2 is a schematic diagram of a distributed detection method for a Socks proxy server provided by an embodiment of the present invention;

3 is a schematic diagram of a distributed detection system module of a Socks proxy server provided by an application example of the present invention;

FIG. 4 is a schematic diagram of the execution flow of the distributed detection system of the Socks proxy server provided by the application example of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

In order to realize the detection of the Socks proxy server with an account and password, and to solve the problem of low recognition rate of the existing Socks proxy server detection method, the present invention provides a detection method of the Socks proxy server, which is used to identify a known ip. Whether the host to be tested is a Socks proxy server, as shown in Figure 1, the detection method includes:

(1) Judge whether the host to be tested is open for service, if so, go to step (2); otherwise, go to step (6);

Optionally, can be combined with the ip of the host to be tested, through network test commands such as ping, tracert and other network test instructions to detect, for example, send the test command "ping-c 3-w 6"+ip to the host to be tested; if the host to be tested If the corresponding information is returned, it means that the host to be tested has opened the service; if the host to be tested does not have any returned information, it means that the host to be tested has not opened the service;

Only when the host to be tested opens services, will it continue to perform subsequent Socks proxy server detection tasks, which can avoid invalid operations and greatly save the time required for detection;

(2) Preset multiple ports for establishing communication connections on the host to be tested;

Not all port numbers 1-65535 in the Internet can be used for Socks to open the proxy. By setting common ports, the detection speed can be saved and the detection efficiency can be improved. In this embodiment of the present invention, the preset ports include 27274, 443,2333,80,8080,4145,6667,9999,8082,9050,3128,8388,8000,8888,8088,1080,9000,53281,54566,808,8081,8118,65103,21071,1080,64312 53281, 54321;

It should be understood that in other application scenarios, it can also be set according to the actual port opening of the detected host;

(3) send the pre-constructed GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port;

The constructed GET request message can be a GET request message with any content conforming to the HTTP protocol. For example, the content of a GET request message constructed according to a blank website is as follows:

'GET/HTTP/1.1\r\nHost:hm.baidu.com\r\n\r\n';

In this request message, hm.baidu.com is the URL of the blank website; the above content is only an exemplary description of the GET request message, and should not be construed as the only limitation of the present invention;

(4) Parse the response information of each port. If it does not contain the first characteristic string and contains the second characteristic string, it is determined that the host to be tested has opened the Socks proxy service through the corresponding port; otherwise, it is determined that the host to be tested has not opened the Socks proxy service. Open the Socks proxy service through the corresponding port;

Wherein, the first feature string and the second feature string are both pre-collected according to the proxy feature, the first feature string is used to exclude the interference of other proxy servers, and the second feature string is used to identify the Socks proxy server;

Specifically, the first characteristic character string and the second characteristic character string are both pre-defined character strings according to the response characteristic of the proxy server to the above-mentioned GET request message;

In an optional implementation manner, the first characteristic string is 'SSH', 'FTP', '\xff\xfd\x18\xff\xfd\xff\xfd#\xff\xfd', 'SMTP' or ' HTTP';

The first characteristic string is a unique string in the response information generated by other proxy servers such as SSH, FTP, SMTP, and HTTP for the GET request message of the HTTP protocol; these proxy servers will interfere with the detection of the Socks proxy server;

In an optional implementation manner, the second characteristic string is '\x05\x00', '\x00[\x00\x00\x00\x00\x00\x00' or 'www.herokucdn.com';

The second characteristic string is a unique string in the returned information generated by the Socks proxy server for the GET request message of the HTTP protocol;

By identifying the host under test that does not contain the first feature string and contains the second feature string as the Socks proxy server, the interference of other proxy servers can be avoided in the process of identifying the Socks proxy server. High recognition accuracy;

It should be noted that the first feature string and the second feature string are not limited to the above-mentioned ones, and other strings that can identify relevant interfering servers can also be incorporated into the first feature string; similarly, other The string that can identify the Socks proxy server can also be incorporated into the second feature string; in addition, the above-mentioned first feature string and second feature string can also be updated according to actual application scenarios;

(5) If the host to be tested has opened the Socks proxy service through any port, identify the host to be tested as a Socks proxy server; otherwise, identify the host to be tested as a non-Socks proxy server;

(6) The detection ends.

The invention uses the GET request message based on the HTTP protocol to detect the Socks proxy server, and identifies whether the host to be tested is a Socks proxy server according to whether the response information of the port on the host to be tested contains a specific character string. A Socks proxy server with an account password or a Socks proxy server without an account password will generate return information corresponding to the GET request message of the HTTP protocol. Therefore, the present invention can identify both a Socks proxy server without an account password and a Socks proxy server without an account password. The Socks proxy server with account and password effectively solves the problem of low recognition rate of the existing Socks proxy server detection method.

In order to further realize the spatial positioning of the Socks proxy server, optionally, the detection method for the Socks proxy server may further include:

If the host to be tested is identified as a Socks proxy server, the spatial location of the host to be tested is obtained according to its ip, so as to realize the physical location of the Socks proxy server, which can provide convenience for the supervision of the Socks proxy server;

Optionally, after identifying the host to be tested as the Socks proxy server, use the API of Baidu Maps, the API of Google Maps, or other APIs to obtain the corresponding latitude and longitude according to its IP bar, so as to obtain the geographic location of the Socks proxy server; further, According to application requirements, the city or other administrative division to which the Socks proxy server belongs can be determined based on the acquired latitude and longitude of the Socks proxy server, and the specific method can also be obtained through the relevant map API.

Based on the detection method for the Socks proxy server, the present invention also provides a distributed detection method for the Socks proxy server, which is used to identify whether a large-scale host to be tested with known ip (that is, a plurality of hosts to be tested with known ip) For the Socks proxy server, the distributed detection method includes:

Divide all the hosts to be tested into multiple sets of hosts to be tested;

For each set of hosts to be tested, the detection method of the above-mentioned Socks proxy server is used to detect the hosts to be tested in turn to identify the Socks proxy server therein; the detection of all sets of hosts to be tested is performed in parallel;

The distributed detection method of the above-mentioned Socks proxy server divides multiple hosts to be tested into sets of hosts to be tested, so that the detection of the hosts to be tested in the set is executed serially, and the detection of multiple sets is executed in parallel, which can effectively improve the performance. Probe rate; under normal circumstances, in order to maximize parallelism, the number of hosts to be tested contained in each set of hosts to be tested is equal or similar.

The present invention also provides a detection system for the Socks proxy server, which is used to realize the detection method for the Socks proxy server. The detection system includes: a pre-judgment module, a port setting module, a transceiver module, an analysis module and an identification module;

The pre-judgment module is used to judge whether the host to be tested has open services, and end the detection when the host to be tested has not opened services;

The port setting module is used to preset a plurality of ports for establishing a communication connection on the host to be tested when the pre-judgment module determines that the host to be tested is open for service;

The transceiver module is used to send the pre-configured GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port;

The parsing module is used to parse the response information of each port, and when the response information does not contain the first feature string and contains the second feature string, it is determined that the host to be tested has opened the Socks proxy service through the corresponding port; and In other cases, it is determined that the host to be tested has not opened the Socks proxy service through the corresponding port;

The identification module is used to identify the host under test as a Socks proxy server when the parsing module determines that the host under test has opened the Socks proxy service through any port; and in other cases, identify the host under test as a non-Socks proxy server;

Wherein, the first feature string and the second feature string are both pre-collected according to the proxy feature, the first feature string is used to exclude the interference of other proxy servers, and the second feature string is used to identify the Socks proxy server;

In this embodiment of the present invention, reference may be made to the descriptions in the foregoing method embodiments for the specific implementation of each module, which will not be repeated here.

According to the fourth aspect of the present invention, a distributed detection system of a Socks proxy server is provided, which is used to realize the distributed detection method of the above-mentioned Socks proxy server. As shown in FIG. 2 , the distributed detection system includes: a database, a central Scheduling nodes and multiple parallel task execution nodes;

The central scheduling node is used to divide all the hosts to be tested into multiple sets of hosts to be tested, and distribute the detection tasks for each set of hosts to be tested to multiple different task execution nodes, so that the sets of hosts to be tested are The detection tasks are executed in parallel;

The task execution node is used to detect the host to be tested in turn by using the detection method of the Socks proxy server to identify the Socks proxy server in the set of hosts to be tested that it has received, and to return the task execution result to the set of hosts to be tested. Send the scheduling result to the center, and store the detection result persistently in the database; the execution result is used to display the success or failure of the task execution, and the detection result includes the IP address, port, and spatial location information of the Socks proxy server;

The central scheduling node is also used to receive the execution result returned by each task execution node, and transfer the failed task from the original task execution node to other task execution nodes to continue execution according to the execution result;

As shown in FIG. 3, in the embodiment of the present invention, the central scheduling node specifically includes: a task distribution module and a failover module;

The task distribution module is used to divide all the hosts to be tested into multiple sets of hosts to be tested, and distribute the detection tasks for each set of hosts to be tested to a plurality of different task execution nodes, so that the sets of hosts to be tested are The detection tasks are executed in parallel;

The failover module is used to receive the execution result returned by each task execution node, and according to the execution result, transfer the failed task from the original task execution node to other task execution nodes to continue execution;

The task execution node includes: a prediction module, a port setting module, a transceiver module, an analysis module, an identification module, a storage module and a return module;

The pre-judgment module is used to judge whether the host to be tested has opened services, and when the host to be tested has not opened services, end the detection of the host to be tested;

The port setting module is used to preset a plurality of ports for establishing a communication connection on the host to be tested when the pre-judgment module determines that the host to be tested is open for service;

The transceiver module is used to send the pre-configured GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port;

The parsing module is used to parse the response information of each port, and when the response information does not contain the first feature string and contains the second feature string, it is determined that the host to be tested has opened the Socks proxy service through the corresponding port; and In other cases, it is determined that the host to be tested has not opened the Socks proxy service through the corresponding port;

The identification module is used to identify the host under test as a Socks proxy server when the parsing module determines that the host under test has opened the Socks proxy service through any port; and in other cases, identify the host under test as a non-Socks proxy server;

The storage module is used to persistently store the detection results in the database;

The return module is used to return the execution result to the central scheduling node;

Wherein, the first feature string and the second feature string are both pre-collected according to the proxy feature, the first feature string is used to exclude the interference of other proxy servers, and the second feature string is used to identify the Socks proxy server;

In this embodiment of the present invention, reference may be made to the descriptions in the foregoing method embodiments for the specific implementation of each module, which will not be repeated here.

In the distributed detection system of the above Socks proxy server, the central scheduling node and the task execution node constitute a Master-Slave (master-slave) model. After scheduling, each task execution node executes the detection task of the Socks proxy server in parallel. This can effectively improve the detection rate of large-scale hosts to be tested.

In an optional embodiment, in the distributed detection system of the Socks proxy server, the task execution node is further configured to re-execute all the detection tasks distributed to it after each preset time interval; It can be comprehensively set according to the execution time of the detection task and the change of the agent to be detected. The execution time of the detection task is determined according to the actual task execution time. The change of the agent to be detected can be set as a constant, and the time interval is the sum of the execution time of the detection task and the constant; for example, For example, it takes 3 hours for a task execution node to execute a task, and the agent host to be detected is updated every 3 hours, because the agent will often change the address and port, then the corresponding time interval is 6 hours, and the task execution time of the task execution node can be predicted through experiments. , but the proxy change is changed by the proxy provider, so a constant can be set in the actual detection, such as a few days, a week or a month, etc.;

The IP and port list of the Socks proxy server will be updated, and the task execution node will re-execute all the detection tasks distributed to it after every preset time interval, which can ensure the validity and real-time of the detection results.

Applications

FIG. 3 shows an application example of the distributed detection system of the Socks proxy server provided by the present invention. Based on the detection system, a schematic diagram of the execution of the distributed detection method of the Socks proxy server provided by the present invention is shown in FIG. 4 . ;

Among them, the Master node is the central scheduling node, and the Slave node is the task execution node; the original data is stored in the database, and the corresponding execution results are stored in the log;

Based on the distributed detection system shown in FIG. 3, the distributed detection method of the Socks proxy server provided by the present invention can be roughly simplified into the following steps:

1. The user submits the task to the Master node -> 2. The Master node receives the task and distributes the task to the Slave node -> 3. The Slave node starts the detection task -> 4. The Slave node obtains the Socks proxy data and records the log -> 5. The Slave node stores the Socks proxy data in the database -> 6. The Slave node returns the execution result to the Master node -> 7. The Master node stores the execution result as a disk file, and determines whether to use a failover mechanism according to the execution result.

Below in conjunction with Fig. 3 and Fig. 4 are described in detail:

(S1) The user collects the target server that needs to be detected, and mainly collects the open IP segment collection of some cloud service providers;

(S2) The user client submits the list of hosts to be detected to the master node, the master node queries the number of online slave nodes, and shards the hosts to be tested. The sharding strategy can dynamically use various customized strategies, including selection polling, randomness, consistency HASH, least frequently used, most recently unused, failover, busy transfer and other strategies. Generally, in order to achieve load balancing, a consistent HASH strategy can be selected. The Master node will distribute the sharding tasks to each Slave according to the selected sharding strategy. node;

(S3) The Master node communicates with the Slave node through heartbeat information. After receiving the fragments sent by the Master node, it starts the proxy detection task, detects these IPs according to the preset ports, and mainly uses network I/O to realize data transmission. The detected information includes ip, the port of the open Socks proxy service, the longitude and latitude information and the city where the Socks proxy server is located;

(S4) Slave node agent can directly obtain ip, open ports and other information when probing. The geographical location of the website, that is, the latitude and longitude cannot be directly detected and obtained. It needs to call the API of Baidu Map or the API of Google Map to obtain it according to the specific IP. The website belongs to Similarly, cities need to call the relevant map API to obtain;

(S5) During the process of the Slave node executing the detection task, if the detection task execution of any host to be tested fails, the task execution node will retry a certain number of times. information;

(S6) The Slave node maintains communication with the Master node while executing the detection task. Once the Slave node fails to execute the task, the Master can monitor the abnormal situation, and the Mater node will transfer the failed detection task from the original execution node to another execution node. , to implement a failover mechanism;

(S7) Each slave node stores the collected data in the database. The database needs to be enabled to allow remote connections. In order to ensure security, the database only allows slave nodes to log in. At the same time, the master node also needs its own database to store task execution logs. , record the execution of the task. Once the task fails, you can analyze it by querying the log, and then determine which specific host to be tested failed the detection task, and restart the detection task, thereby ensuring the integrity of data collection.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, etc., All should be included within the protection scope of the present invention.

Claims (9)

1. a detection method of a Socks proxy server, for identifying whether the known host to be tested of an ip is a Socks proxy server, it is characterized in that, comprising: (1) determine whether the host to be tested is open for service, if so, go to step (2); otherwise, go to step (6); (2) preset a plurality of ports for establishing a communication connection on the host to be tested; (3) send the pre-constructed GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port; (4) Parse the response information of each port, if it does not contain the first feature string and contains the second feature string, then it is determined that the host to be tested has opened the Socks proxy service through the corresponding port; otherwise, it is determined that the The host to be tested has not opened the Socks proxy service through the corresponding port; (5) If the host to be tested has opened the Socks proxy service through any port, then identify the host to be tested as a Socks proxy server; otherwise, identify the host to be tested as a non-Socks proxy server; (6) The detection is over; The first characteristic character string and the second characteristic character string are both character strings pre-collected according to proxy characteristics, the first characteristic character string is used to exclude interference from other proxy servers, and the second characteristic character string String is used to identify the Socks proxy server. 2. the detection method of Socks proxy server as claimed in claim 1 is characterized in that, described first characteristic string is ' SSH ', ' FTP ', '\xff\xfd\x18\xff\xfd\xff\ xfd#\xff\xfd', 'SMTP' or 'HTTP'. 3. The detection method of Socks proxy server as claimed in claim 1, is characterized in that, described second characteristic string is '\x05\x00', '\x00[\x00\x00\x00\x00\x00\ x00' or 'www.herokucdn.com'. 4. The detection method of a Socks proxy server according to any one of claims 1-3, further comprising: if identifying the host to be tested as a Socks proxy server, obtaining the host to be tested according to its ip The spatial location of the Socks proxy server can thus be physically located. 5. a distributed detection method of Socks proxy server, for identifying whether the known large-scale ip host to be tested is a Socks proxy server, it is characterized in that, comprising: Divide all the hosts to be tested into multiple sets of hosts to be tested; For each set of hosts to be tested, the detection method of the Socks proxy server according to any one of claims 1-4 is used to detect the hosts to be tested in turn, so as to identify the Socks proxy server therein; for all the set of guessed hosts The probes are executed in parallel. 6. a detection system of a Socks proxy server, for identifying whether a known ip host to be tested is a Socks proxy server, it is characterized in that, comprising: prejudgment module, port setting module, transceiver module, parsing module and identification module; The pre-judgment module is used to judge whether the host to be tested has opened services, and when the host to be tested has not opened services, end the detection; The port setting module is configured to preset a plurality of ports for establishing a communication connection on the host to be tested when the pre-judgment module determines that the host to be tested is open for service; The transceiver module is used to send the pre-configured GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port; The parsing module is used to parse the response information of each port, and when the response information does not contain the first feature string and contains the second feature string, it is determined that the host to be tested has opened the Socks proxy through the corresponding port service; and in other cases, it is determined that the host to be tested has not opened the Socks proxy service through the corresponding port; The identification module is used to identify the host to be tested as a Socks proxy server when the analysis module determines that the host to be tested has opened the Socks proxy service through any port; and in other cases, to identify the host to be tested. The test host is a non-Socks proxy server; The first characteristic character string and the second characteristic character string are both character strings pre-collected according to proxy characteristics, the first characteristic character string is used to exclude interference from other proxy servers, and the second characteristic character string String is used to identify the Socks proxy server. 7. a distributed detection system of a Socks proxy server, for identifying whether the known large-scale ip host under test is a Socks proxy server, it is characterized in that, comprising: database, central scheduling node and multiple parallel task execution nodes ; The central scheduling node is used to divide all the hosts to be tested into multiple sets of hosts to be tested, and distribute the detection tasks of each set of hosts to be tested to multiple different task execution nodes, so that the The detection tasks are executed in parallel; The task execution node is used for the received host set to be tested, and the detection method of the Socks proxy server described in any one of claims 1-4 is used to detect the host to be tested in turn to identify the host to be tested. wherein the Socks proxy server returns the task execution result to the central scheduling node, and stores the detection result in the database persistently; the execution result is used to display the success or failure of the task execution, and the detection result includes IP address, port, and spatial location information of the Socks proxy server; The central scheduling node is further configured to receive the execution result returned by each task execution node, and according to the execution result, transfer the failed task from the original task execution node to another task execution node to continue execution. 8. The distributed detection system of the Socks proxy server according to claim 7, wherein the central scheduling node comprises: a task distribution module and a failover module; The task distribution module is used to divide all the hosts to be tested into multiple sets of hosts to be tested, and distribute the detection tasks of each set of hosts to be tested to multiple different task execution nodes, so that the The detection tasks are executed in parallel; The failover module is configured to receive the execution result returned by each task execution node, and according to the execution result, transfer the failed task from the original task execution node to another task execution node to continue execution. 9. The distributed detection system of the Socks proxy server according to claim 7 or 8, wherein the task execution node comprises: a pre-judgment module, a port setting module, a transceiver module, an analysis module, an identification module, a storage module modules and return modules; The pre-judgment module is used for judging whether the host to be tested opens services, and when the host to be tested does not open services, ends the detection of the host to be tested; The port setting module is configured to preset a plurality of ports for establishing a communication connection on the host to be tested when the pre-judgment module determines that the host to be tested is open for service; The transceiver module is used to send the pre-configured GET request message based on the HTTP protocol to the host to be tested through each preset port, and obtain the response information of each port; The parsing module is used to parse the response information of each port, and when the response information does not contain the first feature string and contains the second feature string, it is determined that the host to be tested has opened the Socks proxy through the corresponding port service; and in other cases, it is determined that the host to be tested has not opened the Socks proxy service through the corresponding port; The identification module is used to identify the host to be tested as a Socks proxy server when the analysis module determines that the host to be tested has opened the Socks proxy service through any port; and in other cases, to identify the host to be tested. The test host is a non-Socks proxy server; the storage module, configured to persistently store the detection results in the database; the return module, configured to return the execution result to the central scheduling node; The first characteristic character string and the second characteristic character string are both character strings pre-collected according to proxy characteristics, the first characteristic character string is used to exclude interference from other proxy servers, and the second characteristic character string String is used to identify the Socks proxy server.

Images