CN119520539A - A data synchronization method and security gateway device - Google Patents
A data synchronization method and security gateway device Download PDFInfo
- Publication number
- CN119520539A CN119520539A CN202411450816.4A CN202411450816A CN119520539A CN 119520539 A CN119520539 A CN 119520539A CN 202411450816 A CN202411450816 A CN 202411450816A CN 119520539 A CN119520539 A CN 119520539A
- Authority
- CN
- China
- Prior art keywords
- user data
- server
- target user
- search
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application provides a data synchronization method and a security gateway device, wherein the method comprises the steps of sending a first search request to a server, and acquiring a first paging result fed back by the server in response to the first search request. And when the search flag bit of the first paging result is the first flag, sending a first search request to the server again according to the first flag so as to acquire a second paging result. And synchronizing the target user data to the local database when the search flag bit is the second flag. According to the application, when the target user data is requested from the server, the target user data is obtained in batches in a paging mode, and when the residual target user data exists, the target user data is requested from the server again, so that the problem of low data request efficiency caused by one-time request of all the target user data is solved.
Description
Technical Field
The field relates to the technical field of computer security, in particular to a data synchronization method and security gateway equipment.
Background
The LDAP server can acquire the internet surfing behavior data of the user and store the internet surfing behavior data into a directory of the LDAP server. The LDAP protocol (LIGHTWEIGHT DIRECTORY ACCESS PROTOCOL) is a network protocol for accessing and operating directory services, which may allow client applications to access and retrieve directory information in a standard manner.
In order to facilitate management of the internet surfing behavior data of the user, the security gateway device may search the LDAP server for the internet surfing behavior data of the preset condition through the LDAP protocol. However, the number of users is too large, so that the data volume of the internet surfing behavior data generated by the users is too large. The large amount of internet surfing behavior data can cause the security gateway device to search for internet surfing behavior data at a slower speed when performing a full amount of search, thereby causing inefficiency in synchronizing the internet surfing behavior data.
Disclosure of Invention
In order to solve the problem of low efficiency caused by excessive data when the security gateway equipment synchronizes the internet surfing behavior data of the user. In a first aspect, the present application provides a data synchronization method, the method comprising:
Generating a first search request for requesting target user data;
the first search request is sent to a server to obtain a first paging result fed back by the server, wherein the first paging result comprises target user data and a search flag bit, the search flag bit is a first flag or a second flag, the first flag is used for indicating that the server does not send all target user data, and the second flag is used for indicating that the server has sent all target user data;
When the search flag bit is a first flag bit, a first search request is sent to the server again according to the first flag bit so as to obtain a second paging result fed back by the server, wherein the second paging result comprises other target user data except the target user data in the first paging result;
synchronizing the target user data to the local database when the search flag bit is the second flag
In some embodiments, the step of resending the first search request to the server, the method further comprising:
In the process of acquiring a second paging result fed back by the server, synchronizing target user data in the first paging result to the local database;
and after the synchronization of the target user data in the first paging result is finished, synchronizing the target user data of the second paging result to the local database.
In some embodiments, the step of generating the first search request includes:
setting a first search rule and a paging data quantity, wherein the first search rule is a search rule of the target user data, and the paging data quantity is used for indicating the quantity of the target user data contained in a paging result;
And generating the first search request according to the first search rule and the paging data quantity.
In some embodiments, the step of synchronizing the target user data to a local database comprises:
Creating a first data set and a second data set in the local database according to preset conditions;
Detecting a data type of the target user data, the data type comprising individual user data and non-individual user data;
Synchronizing the target user data to the first data set if the target user data is individual user data;
And if the target user data is non-individual user data, synchronizing the target user data to the second data set.
In some embodiments, after the step of sending the first search request to the server, the method further comprises:
setting a limit number of times of sending the first search request;
When the number of times of sending the first search request is greater than or equal to the limit number, adding a limit mark in the first search request to obtain a first limit request;
sending the first limiting request to the server to obtain a paging termination result fed back by the server;
and synchronizing the target user data of the paging termination result and the target user data acquired before the paging termination result to the local database respectively.
In some embodiments, after the step of sending the first search request to the server, the method further comprises:
Recording the search duration of the server;
When the search time length is greater than or equal to a preset time length threshold, if the first paging result is not received, generating a timeout request;
And sending a timeout request to the server to acquire a timeout search result fed back by the server, wherein the timeout search result is generated by the server according to the currently searched target user data in response to the timeout request.
In some embodiments, the first search request is sent to a server, the method further comprising:
sending a connection request to a server to establish a communication channel between the security gateway device and the server;
through the communication channel, communication binding is carried out with the server according to the user name and the password information corresponding to the user name;
after the server is bound in communication, the first search request is sent to the server.
In some embodiments, after the step of synchronizing the target user data to the local database, the method further comprises:
Deleting the first paging result and the second paging result;
And after deleting the first paging result and the second paging result, releasing the binding state of the security gateway equipment and the server.
In some embodiments, if the first paging result does not include the target user data, the method further comprises:
And generating prompt information according to the first search rule, wherein the prompt information is used for representing that the target user data is not synchronized to the local database.
In a second aspect, the present application provides a security gateway device comprising a data synchronization module configured to:
Generating a first search request for requesting target user data;
the first search request is sent to a server to obtain a first paging result fed back by the server, wherein the first paging result comprises target user data and a search flag bit, the search flag bit is a first flag or a second flag, the first flag is used for indicating that the server does not request all the target user data, and the second flag is used for indicating that the server has requested all the target user data;
When the search flag bit is a first flag bit, a first search request is sent to the server again according to the first flag bit so as to obtain a second paging result fed back by the server, wherein the second paging result comprises other target user data except the target user data in the first paging result;
and synchronizing the target user data to a local database when the search flag bit is the second flag.
As can be seen from the above technical solutions, the present application provides a data synchronization method and a security gateway device, according to the method, a first search request is sent to a server, and a first paging result fed back by the server in response to the first search request is obtained. And when the search flag bit of the first paging result is the first flag, sending a first search request to the server again according to the first flag so as to acquire a second paging result. And synchronizing the target user data to the local database when the search flag bit is the second flag. According to the application, when the target user data is requested from the server, the target user data is obtained in batches in a paging mode, and when the residual target user data exists, the target user data is requested from the server again, so that the problem of low data request efficiency caused by one-time request of all the target user data is solved.
Drawings
In order to more clearly illustrate the technical solution of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a flow chart of a method for synchronizing data according to some embodiments of the present application;
FIG. 2 is a flow chart illustrating the generation of different flags according to the amount of paged data in some embodiments of the present application;
FIG. 3 is a flow chart of generating a second page splitting result in some embodiments of the application;
FIG. 4 is a flow chart of batch synchronization data according to some embodiments of the application.
Detailed Description
For the purposes of making the objects and embodiments of the present application more apparent, an exemplary embodiment of the present application will be described in detail below with reference to the accompanying drawings in which exemplary embodiments of the present application are illustrated, it being apparent that the exemplary embodiments described are only some, but not all, of the embodiments of the present application.
It should be noted that the brief description of the terminology in the present application is for the purpose of facilitating understanding of the embodiments described below only and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.
The terms "first," "second," "third," and the like in the description and in the above drawings are used for distinguishing between similar or similar objects or entities and not necessarily for limiting a particular order or sequence, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.
The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.
The LDAP server can provide various fields of user management, authority control, group policy and other services for individual users, enterprises, institutions and other organizations. For example, the LDAP server may obtain internet surfing behavior data of each user in the organization in the network domain, so as to perform internet surfing behavior management on the users in the organization according to the monitored internet surfing behavior data.
In the process of online behavior management, the LDAP server can acquire online behavior data of a user and store the online behavior data into a database of the LDAP server, and in order to facilitate classification of different data types, the server database is provided with a plurality of storage catalogs, and the different types of online behavior data can be stored under the corresponding storage catalogs according to different user information. The LDAP protocol (LIGHTWEIGHT DIRECTORY ACCESS PROTOCOL) is a network protocol for accessing and operating directory services, which may allow client applications to access and retrieve directory information in a standard manner.
In order to improve network security, the security gateway device can monitor the internet surfing behavior data of the user, and therefore, the security gateway device can request the internet surfing behavior data from the LDAP server through the LDAP protocol, and timely remove security risk data existing in the internet surfing behavior data, so that the security of the user using the network is improved.
However, as the number of users increases, the data volume of the internet surfing behavior data generated by the users is also huge. The large amount of internet surfing behavior data can cause the security gateway device to search for internet surfing behavior data at a slower speed when performing a full amount of search, thereby causing inefficiency in synchronizing the internet surfing behavior data.
In order to solve the problem of inefficiency caused by excessive data when the security gateway device synchronizes the internet surfing behavior data of the user, some embodiments of the present application provide a data synchronization method applied to the security gateway device, that is, the security gateway device can execute the data synchronization method. Fig. 1 is a flowchart of a data synchronization method according to some embodiments of the present application. Referring to fig. 1, the method includes:
and S100, generating a first search request.
The LDAP server may be communicatively coupled to a plurality of user's terminal devices, and may generate user data when the user performs an interactive operation using the terminal devices. Depending on the interaction options, the user data may comprise various types, e.g. for network interactions, the user data may comprise internet behavior data, for information registration interactions, the user data may comprise user information data, for communication interactions, the user data may also comprise user communication information. In the present application, only internet behavior data is exemplified as user data, and it should be understood that in practical applications, data of different categories may be used as user data.
To facilitate network security management of user data, the security gateway device may generate a first search request. The first search request is for requesting target user data from a server. The target user data is user data that meets the first search criteria. Because the data volume of the user data is huge, in order to facilitate the improvement of the management efficiency, the network security management may set the first search rule to reduce the user data range of the network security management.
Based on the above scenario, the security gateway device may further set a first search rule, which is a search rule for target user data, for example, the security gateway device wants to detect that the IP address is user data of city a, and thus may set the first search rule to "IP address: city a" and generate the first search request according to the first search rule.
In some embodiments, because the number of users is greater, in order to facilitate improving the searching efficiency and the data synchronization efficiency, the security gateway device may further make the server page the target user data through the first searching request, that is, sequentially return the paging result through the server, where the paging result includes a preset number of target user data, so as to improve the searching efficiency by searching and returning the target user data in multiple batches. To this end, the security gateway device may set the amount of paging data and generate a first search request according to the first search rule and the amount of paging data. The paging data amount is used for indicating the amount of target user data contained in the paging result.
And S200, sending the first search request to a server to acquire a first paging result fed back by the server.
After generating the first search request, the security gateway device may send the first search request to the server. The server may search the server database for target user data in response to the first search request and generate a first paging result based on the target user data. The first paging result is a paging result sent to the security gateway device by the server for the first time, and the first paging result comprises a plurality of target user data and search zone bits.
The server may generate the first paging result according to the amount of the paging data in the first search request, for example, when the amount of the paging data is 100 pieces, the server may respond to the first search request, and in the search process, fill the paging result according to the target user data obtained by the search, and use 100 as the amount of the paging data, that is, generate one paging result every time 100 pieces of the target user data are searched. When the search starts, the server may generate a first paging result according to the current paging result and transmit the first paging result to the security gateway device when the number of target user data in the paging result is equal to the number of paging data.
After receiving the first paging result, the security gateway device may determine, according to the search flag bit, whether all target user data is requested in the server database. The search flag bit may be a first flag indicating that the server has not transmitted all of the target user data or a second flag indicating that the server has requested all of the target user data.
The search flag bit is generated by the server according to the target user data and the paging data quantity in the server database. As shown in fig. 2, when the server responds to the first search request, if the number of target user data is greater than the number of paging data, it is indicated that there is remaining target user data in the server database when the server generates the first paging result. In order to improve the synchronization efficiency of the target user data, the server needs to send the target user data feedback in the first paging result to the security gateway device first, but since there is still residual target user data in the server database. For example, when the number of pieces of paging data is 20, the server obtains 50 pieces of target user data according to the first search request, and the server may first generate the first paging result according to 20 pieces of target user data among them, and the remaining 30 pieces of target user data.
It should be noted that, during the searching process of the server, the target user data is obtained by searching in sequence, so that when the target user data is searched for the bottom 20 items, the server can continue searching based on the first search request. If new target user data is searched, the existence of the residual target user data is indicated, the server generates a first mark in the first paging result and sends the first paging result to the security gateway device, so that the security gateway device can execute processes such as caching, synchronization, backup and the like on the target user data in the first paging result. The security gateway device again transmits a search request to the server according to the first flag of the search flag bit to request the remaining target user data.
With continued reference to fig. 2, if the number of target user data is equal to the number of paging data, it is indicated that the target user data in the server database may just generate the first paging result, and at this time, there is no remaining target user data in the server database, the server may generate a second flag in a search flag bit of the first paging result to prompt the security gateway device, where there is no remaining target user data in the server database.
If the number of the target user data is smaller than the number of the paging data, the target user data cannot completely fill the first paging result, and the server can generate the first paging result according to the current target user data and generate a second mark in a search mark bit of the first paging result. For example, when the number of pieces of paging data is 20, that is, the first paging result may store 20 pieces of target user data, at this time, the server obtains 10 pieces of target user data according to the first search request, which indicates that the target user data of the server database is insufficient to reach 20 pieces, at this time, the server may directly generate the first paging result according to the 10 pieces of target user data.
In some embodiments, the security gateway device may also record the search duration of the server in order to determine that the search target user data is terminated. When the search time length is greater than or equal to the preset time length threshold, if the security gateway device does not receive the first paging result, a timeout request can be generated, and the timeout request is used for prompting the server to request the target user data timeout. The security gateway device sends a timeout request to the server, the server responds to the timeout request, the server terminates searching for target user data, generates a timeout search result according to the target user data obtained by current searching, and sends the timeout search result to the security gateway device.
In some embodiments, the security gateway device may also replace the step of recording the search duration of the server by setting a search duration threshold of the server. The security gateway device may send a search duration request to the server, the search duration request to set a search duration threshold for the server to search for the target user data. When the search time length of the server is greater than or equal to the search time length threshold, if new target user data is not searched, stopping searching the target user data, and generating a first paging result according to the target user data obtained by current searching, so that the time length monitoring process of the security gateway equipment to the server is omitted, the equipment power consumption is reduced, and the equipment operation efficiency of the security gateway equipment is improved.
And S300, when the search flag bit is the first flag, a first search request is sent to the server again according to the first flag so as to acquire a second page splitting result fed back by the server.
When the search flag bit is the first flag, it is indicated that there is remaining target user data in the server database. The security gateway device may send a first search request again to the server according to the first flag to request remaining target user data from the server.
The server may again respond to the first search request and generate a second split result based on the target user result. The second paging result is a paging result generated after the first paging result and comprises other target user data except the target user data in the first paging result. The second paging result is the same as the paging data amount of the first paging result.
After the server generates the second splitting result, if there is still remaining target user data, the server may further generate a first flag again in the second splitting result, so that the security gateway device sends the first search request to the server again according to the first flag until all the target user data in the server database is requested. To this end, a plurality of second paging results may be included after the first paging result according to the number of target user data. For example, as shown in fig. 3, when the number of pieces of paging data is 20, the server obtains 50 pieces of target user data according to the first search request, and the server may first generate the first paging result according to 20 pieces of target user data. When the server receives the first search request again, a second page splitting result is generated according to 20 items of target user data in the rest 30 items of target user data. At this time, there is still 10 pieces of target user data, and the server generates a first flag in the search flag bit of the second page splitting result, so that the security gateway device sends the first search request again to request another second page splitting result generated according to the last 10 pieces of target user data.
And S400, synchronizing the target user data to a local database when the search flag bit is the second flag.
When the search flag is the second flag, indicating that the target user data in the server database has been completely requested, the security gateway device may perform synchronization on the target user data to cache all of the target user data to the local database of the security gateway device.
The security gateway device may synchronize target user data in each paging result to the local database in a unified manner after obtaining all the paging results. In order to improve the efficiency of data synchronization, as shown in fig. 4, after receiving the first paging result fed back by the server, the security gateway device may further send the first search request to the server again, so as to synchronize the target user data in the first paging result to the local database in the process of obtaining the second paging result fed back by the server, so as to execute the first data synchronization. Therefore, the server can synchronize the target user data in the first paging result while generating the second paging result according to the residual target user data in response to the retransmitted first search request, so that the time for waiting for the residual target user data is saved, and the data synchronization efficiency is improved.
After the target user data synchronization in the first paging result is finished, as shown in fig. 4, the security gateway device may synchronize the target user data of the second paging result to the local database to perform the second data synchronization. If there are multiple second page splitting results, in the process of sending the first search request to the server to obtain another second page splitting result fed back by the server, the target user data of the second page splitting result already fed back by the server currently can be synchronized according to the synchronization manner in the above embodiment.
In some embodiments, the transmission of the first search request multiple times in a short time is easily determined as an abnormal request behavior, and therefore, the security gateway device may further set a limit number of times of the first search request after transmission to re-transmit the first search request at an interval of a preset time period after the limit number of times, or stop transmitting the first search request. The security gateway device may record the number of times when the first search request is transmitted. When the number of times of sending the first search request is greater than or equal to the limit number, adding a limit mark in the first search request to obtain a first limit request, and sending the first limit request to the server. The first limit request is used to instruct the server to terminate the request for the target user data after the current round of requesting the paging result, even though the server further includes the remaining target user data, the request is not performed on the remaining target user data due to the limit of the number of search requests. The server may generate a termination paging result from the target user data in response to the first restriction request and send termination paging result feedback to the security gateway device.
In order to limit the security gateway device from continuing to send the first search request, the server may directly generate the second flag in the search flag bit in generating the termination paging result, that is, the search flag bit of the termination paging result is the second flag. The security gateway device may synchronize the target user data of the termination paging result and the target user data acquired before the termination paging result to the local database, respectively, according to the second flag.
In some embodiments, the server may further determine the first flag or the second flag according to a flag value, where the flag value is a preset value of the search flag bit, and the flag value may be a cookie value, for example. When the server receives the first search request and begins to perform searching for target user data, the cookie value may be null to indicate that the target user data is not currently searched. When the server searches for the target user data, the cookie value may be updated to a non-null value, at which time the server may generate a first flag based on the non-null value to indicate that there is remaining target user data outside of the current paging result. When the cookie value is updated to a null value after the server responds to one of the first search requests, the server may generate a second flag to indicate that the request for all target user data is completed.
In some embodiments, the target classification data may also be class synchronized when the security gateway device synchronizes the target user data. Thus, the security gateway device may create the first data set and the second data set in the local database according to preset conditions. The preset condition corresponds to a data type of the target user data, and the security gateway device may detect the data type of the target user data after accepting the target user data. Wherein the data types include individual user data and non-individual user data.
If the target user data is individual user data, synchronizing the target user data to the first data set. The target user data can be represented by a user result pointer, and the security gateway device can search the directory function of the target user data fed back by the server by calling the ldap_search_st (), request to acquire the target user data under the corresponding directory path, and return a user result pointer res_entry after the function is successfully executed. And fetching the user_entry by circularly traversing the res_entry pointer, and storing the user_entry into the first data group so as to complete the synchronization process of the target user data.
If the target user data is non-individual user data, synchronizing the target user data to the second data set. The security gateway device may obtain the current path of the target user data by calling ldap_get_dn (ld, entry), typically the current path is a string that is hierarchically consistent with the domain server.
For example, the current path may be dn=ou=ou502, ou=1024_ou, ou=ngfw, dc= NGFW, dc=com, the domain hierarchy organization corresponding to the current path is/ngfw.com/NGFW/1024_ou/OU 502, the security gateway device needs to parse dn and assemble the dn according to the hierarchy, after assembling, the path of the target user data may be stored in the second data set, and the completed target user data may be synchronized to the second data set.
In another embodiment, the non-individual user data may further include path information associated with the first search rule and may be stored according to the path information associated with the first search rule. For example, when the first search rule is to search for user data of "school a", the path information corresponding to "student E reading from school a" is school a/grade B/class C/group D/student E, etc., and the security gateway device may store the target user data corresponding to student E according to the path information.
In some embodiments, the security gateway device also needs to establish a communication connection with the server before sending the first search request to the server. To this end, the security gateway device may send a connection request to the server to establish a communication channel between the security gateway device and the server. The communication channel may be used for data interaction between the security gateway device and the server, e.g. the server may send user data to the security gateway device via the communication channel, and the security gateway device may upload corresponding interaction data to the server.
In order to improve the security of data transmission, the security gateway device needs to pass the authentication of the server, and therefore, the security gateway device can be in communication binding with the server through a communication channel according to the user name and the password information corresponding to the user name. The user name is the user name of the security gateway device. After passing the authentication of the server, the security gateway device completes the communication binding with the server. After the communication binding is completed, the server may send binding information to the gateway security device, and the gateway security device may send the first search request to the server according to the binding information.
In some embodiments, when the security gateway device synchronizes the target user data in each paging result fed back by the server, the security gateway device may delete the first paging result and the second paging result, so as to increase the storage space of the security gateway device and avoid the risk of data leakage. After deleting the first paging result and the second paging result, if the security gateway device does not have other data request tasks, the binding state of the security gateway device and the server can be released.
In some embodiments, if the server does not request the target user data according to the first search rule of the first search request, it is indicated that there is no target user data in the server database. The server may generate a first paging result that does not include the target user data based on the blank data information. And when the security gateway equipment receives the first paging result, prompt information can be generated according to a first search rule, and the prompt information is used for representing unsynchronized target user data to the local database. The security tester may change the first search rule according to the prompt information to generate another search request.
In some embodiments, the server may create a page_control according to the first search request, where the page_control is a control page, and the server may generate the paging result according to the control page and the target user data obtained by the search. After the server generates the page_control, it is necessary to use the page_control as a control parameter for the page search, and call the ldap_search_ext_s function to search for the target user data to be synchronized based on the control parameter.
Ldap_search_ext/u the s function may be ：LDAP*ld,const char*base,int scope,const char*filter,LDAPControl**serverctrls,LDAPControl**clientctrls,struct l_timeval*timeout,struct l_timeval*sizelimit,LDAPMessage**res.
Where ld is a pointer to a server, which in this embodiment may be an LDAP server, and thus ld may represent an LDAP connection. base, base of search DN (Distinguished Name) to indicate the starting point of searching for target user data. SCOPE, which may be LDAP_SCOPE_BASE (searching only the BASE object), LDAP_SCOPE_ ONELEVEL (searching only the direct child object), or LDAP_SCOPE_ SUBTREE (searching the BASE object and all its child data). filter, search filter, is used for filtering the user data according to the first search rule, in order to get the goal user data. serverctrls a pointer to a server control array for directing the server to search for target user data. clientctrls pointers to client control arrays that can be used to affect server behavior, e.g., modify parameters of a page search to change the search policy of the server. timeout-a pointer to the l_ timeval structure defines the timeout time for the search operation. The value of Timeout is null, which indicates that the duration of searching for the target user data is Timeout. sizelimit a pointer to the l_ timeval structure defining the number of search requests sent. res, a pointer to LDAPMESSAGE structures, for receiving the searched target user data. After the function is successfully executed and returned, the res pointer contains target user data returned by the server, and the res pointer can be analyzed to obtain a synchronization result of the target user data.
Some embodiments of the present application further provide a security gateway device, where the security gateway device includes a data synchronization module configured to perform a data synchronization method disclosed in the foregoing embodiments, and the method includes:
and S100, generating a first search request.
Wherein the first search request is for requesting target user data.
And S200, sending the first search request to a server to acquire a first paging result fed back by the server.
The first paging result comprises target user data and a search flag bit, wherein the search flag bit is a first flag or a second flag, the first flag is used for indicating that the server does not request all the target user data, and the second flag is used for indicating that the server has requested all the target user data.
And S300, when the search flag bit is a first flag, a first search request is sent to the server again according to the first flag so as to acquire a second page splitting result fed back by the server.
The second paging result includes other target user data except the target user data in the first paging result.
And S400, synchronizing the target user data to a local database when the search flag bit is the second flag.
As can be seen from the above technical solutions, the present application provides a data synchronization method and a security gateway device, according to the method, a first search request is sent to a server, and a first paging result fed back by the server in response to the first search request is obtained. And when the search flag bit of the first paging result is the first flag, sending a first search request to the server again according to the first flag so as to acquire a second paging result. And synchronizing the target user data to the local database when the search flag bit is the second flag. According to the application, when the target user data is requested from the server, the target user data is obtained in batches in a paging mode, and when the residual target user data exists, the target user data is requested from the server again, so that the problem of low data request efficiency caused by one-time request of all the target user data is solved.
It should be noted that the above embodiments are merely for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solution described in the above embodiments may be modified or some or all of the technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present application.
The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. The illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the present disclosure and to enable others skilled in the art to best utilize the embodiments.
Claims (10)
1. A method of data synchronization, applied to a security gateway device, comprising:
Generating a first search request for requesting target user data;
the first search request is sent to a server to obtain a first paging result fed back by the server, wherein the first paging result comprises target user data and a search flag bit, the search flag bit is a first flag or a second flag, the first flag is used for indicating that the server does not send all target user data, and the second flag is used for indicating that the server has sent all target user data;
When the search flag bit is a first flag bit, a first search request is sent to the server again according to the first flag bit so as to obtain a second paging result fed back by the server, wherein the second paging result comprises other target user data except the target user data in the first paging result;
and synchronizing the target user data to a local database when the search flag bit is the second flag.
2. The data synchronization method according to claim 1, characterized by the step of resending the first search request to the server, the method further comprising:
In the process of acquiring a second paging result fed back by the server, synchronizing target user data in the first paging result to the local database;
and after the synchronization of the target user data in the first paging result is finished, synchronizing the target user data of the second paging result to the local database.
3. The data synchronization method according to claim 1, wherein the step of generating the first search request includes:
setting a first search rule and a paging data quantity, wherein the first search rule is a search rule of the target user data, and the paging data quantity is used for indicating the quantity of the target user data contained in a paging result;
And generating the first search request according to the first search rule and the paging data quantity.
4. The data synchronization method of claim 1, wherein the step of synchronizing the target user data to the local database comprises:
Creating a first data set and a second data set in the local database;
Detecting a data type of the target user data, the data type comprising individual user data and non-individual user data;
Synchronizing the target user data to the first data set if the target user data is individual user data;
And if the target user data is non-individual user data, synchronizing the target user data to the second data set.
5. The data synchronization method according to claim 1, wherein after the step of sending the first search request to the server, the method further comprises:
setting a limit number of times of sending the first search request;
When the number of times of sending the first search request is greater than or equal to the limit number, adding a limit mark in the first search request to obtain a first limit request;
sending the first limiting request to the server to obtain a paging termination result fed back by the server;
and synchronizing the target user data of the paging termination result and the target user data acquired before the paging termination result to the local database respectively.
6. The data synchronization method according to claim 1, wherein after the step of sending the first search request to the server, the method further comprises:
Recording the search duration of the server;
When the search time length is greater than or equal to a preset time length threshold, if the first paging result is not received, generating a timeout request;
And sending a timeout request to the server to acquire a timeout search result fed back by the server, wherein the timeout search result is generated by the server according to the currently searched target user data in response to the timeout request.
7. The data synchronization method of claim 1, wherein the first search request is sent to a server, the method further comprising:
sending a connection request to a server to establish a communication channel between the security gateway device and the server;
through the communication channel, communication binding is carried out with the server according to the user name and the password information corresponding to the user name;
after the server is bound in communication, the first search request is sent to the server.
8. The data synchronization method of claim 7, wherein after the step of synchronizing the target user data to the local database, the method further comprises:
Deleting the first paging result and the second paging result;
And after deleting the first paging result and the second paging result, releasing the binding state of the security gateway equipment and the server.
9. The method of claim 3, wherein if the first paging result does not include the target user data, the method further comprises:
And generating prompt information according to the first search rule, wherein the prompt information is used for representing that the target user data is not synchronized to the local database.
10. A security gateway device, the security gateway device comprising a data synchronization module configured to:
Generating a first search request for requesting target user data;
the first search request is sent to a server to obtain a first paging result fed back by the server, wherein the first paging result comprises target user data and a search flag bit, the search flag bit is a first flag or a second flag, the first flag is used for indicating that the server does not request all the target user data, and the second flag is used for indicating that the server has requested all the target user data;
When the search flag bit is a first flag bit, a first search request is sent to the server again according to the first flag bit so as to obtain a second paging result fed back by the server, wherein the second paging result comprises other target user data except the target user data in the first paging result;
and synchronizing the target user data to a local database when the search flag bit is the second flag.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411450816.4A CN119520539A (en) | 2024-10-16 | 2024-10-16 | A data synchronization method and security gateway device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411450816.4A CN119520539A (en) | 2024-10-16 | 2024-10-16 | A data synchronization method and security gateway device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119520539A true CN119520539A (en) | 2025-02-25 |
Family
ID=94660490
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411450816.4A Pending CN119520539A (en) | 2024-10-16 | 2024-10-16 | A data synchronization method and security gateway device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119520539A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130173655A1 (en) * | 2012-01-04 | 2013-07-04 | International Business Machines Corporation | Selective fetching of search results |
| CN110765158A (en) * | 2019-10-31 | 2020-02-07 | 北京明略软件系统有限公司 | Paging query method, system and device |
| CN118606580A (en) * | 2024-06-20 | 2024-09-06 | 上海数禾信息科技有限公司 | A server-side paging processing method, device, computer equipment and storage medium |
-
2024
- 2024-10-16 CN CN202411450816.4A patent/CN119520539A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130173655A1 (en) * | 2012-01-04 | 2013-07-04 | International Business Machines Corporation | Selective fetching of search results |
| CN110765158A (en) * | 2019-10-31 | 2020-02-07 | 北京明略软件系统有限公司 | Paging query method, system and device |
| CN118606580A (en) * | 2024-06-20 | 2024-09-06 | 上海数禾信息科技有限公司 | A server-side paging processing method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110351283B (en) | Data transmission method, device, equipment and storage medium | |
| CN100414539C (en) | Method for transmitting and downloading streaming data | |
| CN100533435C (en) | Media for accessing network | |
| KR100754907B1 (en) | How to update the search engine index | |
| CN112788126B (en) | Resource downloading method, device, server and storage medium | |
| CN104144098B (en) | Information push method, system and push server equipment | |
| CN103685590B (en) | Obtain the method and system of IP address | |
| EP2924941A1 (en) | Method and device for preventing service illegal access | |
| CN106533944B (en) | Distributed API gateway, management method and management system | |
| US11416564B1 (en) | Web scraper history management across multiple data centers | |
| CN111427613B (en) | Application program interface API management method and device | |
| CN103501241B (en) | Method, device and system for cleaning useless files | |
| CN109617780A (en) | Access method, apparatus, terminal device and the machine readable storage medium of network | |
| US11297131B2 (en) | Method and apparatus for multi-vendor GTM fabric | |
| CN105704246A (en) | Network distribution device and method based on SDN architecture | |
| CN111585786A (en) | Realization of secret-free building method of big data cluster | |
| JP4009591B2 (en) | Domain naming system (DNS) for accessing databases | |
| Allman et al. | A scalable system for sharing internet measurements | |
| CN108683651A (en) | A kind of single-point logging method, server-side and system | |
| CN114301988B (en) | Distributed calling method, device, storage medium and electronic device | |
| KR20040012311A (en) | Contents Sharing Service System and Method by Presence Information in Wireless Network | |
| CN113067710A (en) | Online user query method and device, computer equipment and storage medium | |
| CN119520539A (en) | A data synchronization method and security gateway device | |
| CN114157435B (en) | Document processing method and device based on blockchain technology and client | |
| CN113612811B (en) | Method, system, equipment and medium for client mounting in multiple channels |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |