CN108182203A - Data checking method and device - Google Patents

Abstract

The invention provides a data checking method and a device, and the method can comprise the following steps: reading a required view from a target database; obtaining a first data result set corresponding to the view from the target database; and checking a second data result set corresponding to the first data result set in a local database according to the first data result set. And each time of data checking, automatically reading a required view from the corresponding target database, and acquiring a standard first data result set from the target database through the required view, so that checking a second data result set corresponding to the first data result set in the local database can be completed according to the first data result set. The whole data checking process does not need manual participation, can be automatically realized, and improves the user experience.

Description

A data verification method and device

technical field

The invention relates to the field of computer technology, in particular to a data checking method and device.

Background technique

Nowadays, different business systems mostly use different databases, and different databases may need to share the same part of data. Therefore, in order to ensure the consistency of related data in different databases, it is necessary to check the related data in different databases.

At present, each time data checking is performed, technicians use database connection tools to connect to the databases to be checked one by one, and then extract relevant data from each database and check them one by one. It can be seen that manual intervention is required for each data verification, and the user experience is poor.

Contents of the invention

The embodiment of the present invention provides a data checking method, which can automatically perform data checking and improve user experience.

In a first aspect, an embodiment of the present invention provides a data checking method, including:

Read the desired view from the target database;

Acquiring a first data result set corresponding to the view from the target database;

According to the first data result set, check the second data result set corresponding to the first data result set in the local database.

Preferably,

After the acquisition of the first data result set corresponding to the view from the target database, and according to the first data result set, checking the first data result set corresponding to the first data result set in the local database Before the second data result set, further include:

creating a temporary table in said local database;

inserting the first data result set into the temporary table;

The checking the second data result set corresponding to the first data result set in the local database according to the first data result set includes:

Checking the second data result set in the local database by calling and executing SQL according to the first data result set inserted in the temporary table.

Preferably,

After checking the second data result set corresponding to the first data result set in the local database according to the first data result set, further comprising:

When it is checked that the second data result set is inconsistent with the first data result set, update the second data result set so that the updated second data result set is consistent with the first data result set unanimous.

Preferably,

The updating the second data result set includes:

According to the checking result, determine the newly added data added in the first data result set;

Synchronizing the newly added data into the second data result set.

Preferably,

The updating the second data result set includes:

According to the checking result, determine the modified data in the first data result set after modification;

changing target data in the second data result set to the changed data, wherein the target data corresponds to the changed data;

Preferably,

The updating the second data result set includes:

According to the result of checking, determine the deleted deleted data in the first data result set;

and deleting the deleted data included in the second data result set.

Preferably,

Before reading the required view from the target database, further include:

According to the preset checking period, the connection with the target database is established periodically according to the preset connection information of the target database.

Preferably,

After checking the second data result set corresponding to the first data result set in the local database according to the first data result set, further comprising:

When it is checked that the second data result set is inconsistent with the first data result set, generating feedback information for indicating that the data verification is inconsistent;

Send the feedback information to at least one preset target address.

In a second aspect, an embodiment of the present invention provides a data checking device, including:

View reading unit, data acquisition unit and data checking unit, wherein,

The view reading unit is used to read the required view from the target database;

The data acquisition unit is configured to acquire a first data result set corresponding to the view from the target database;

The data checking unit is configured to check a second data result set corresponding to the first data result set in the local database according to the first data result set.

Preferably,

further includes: creating a unit;

The creating unit is configured to, after the data obtaining unit obtains the first data result set corresponding to the view from the target database, and after the data checking unit according to the first data result set, Before checking the second data result set corresponding to the first data result set in the local database, create a temporary table in the local database; insert the first data result set into the temporary table;

The data checking unit is configured to check the second data result set in the local database by calling and executing SQL according to the first data result set inserted in the temporary table.

Preferably,

Further comprising: a connection unit;

The connection unit is configured to, before the view reading unit reads the required view from the target database, according to the preset checking cycle, periodically establish a view according to the preset connection information of the target database. A connection to the target database.

Preferably,

further comprising: an update unit;

The updating unit is configured to update the second data result set when the data checking unit checks that the second data result set is inconsistent with the first data result set, so that the updated first data result set The second data result set is consistent with the first data result set.

Preferably,

The updating unit is configured to determine the newly added data in the first data result set according to the checking result; and synchronize the added data to the second data result set.

Preferably,

The update unit is configured to determine the modified data in the first data result set according to the checking result; change the target data in the second data result set to the modified data, wherein the target data Corresponding to said modified data;

Preferably,

The updating unit is configured to determine the deleted data in the first data result set according to the checking result; and delete the deleted data included in the second data result set.

The embodiment of the present invention provides a data checking method and device. Each time data checking is performed, the required view will be automatically read from the corresponding target database, and the standard view can be obtained from the target database through the required view. In this way, according to the first data result set, the check of the second data result set corresponding to the first data result set in the local database can be completed. The entire data verification process can be realized automatically without manual participation, which improves the user experience.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are For some embodiments of the present invention, those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a flow chart of a data checking method provided by an embodiment of the present invention;

Fig. 2 is a flow chart of a data checking method provided by another embodiment of the present invention;

Fig. 3 is a schematic structural diagram of a data checking device provided by an embodiment of the present invention;

Fig. 4 is a schematic structural diagram of a data checking device provided by another embodiment of the present invention;

Fig. 5 is a schematic structural diagram of a data checking device provided by another embodiment of the present invention;

Fig. 6 is a schematic structural diagram of a data checking device provided by another embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection of the present invention. scope.

As shown in Figure 1, an embodiment of the present invention provides a data checking method, which may include the following steps:

Step 101: Read the required view from the target database.

Step 102: Obtain a first data result set corresponding to the view from the target database.

Step 103: Check the second data result set corresponding to the first data result set in the local database according to the first data result set.

The embodiment of the present invention provides a data checking method. Each time data checking is performed, the required view will be automatically read from the corresponding target database, and the standard first view can be obtained from the target database through the required view. A data result set. In this way, the second data result set corresponding to the first data result set in the local database can then be checked according to the first data result set. The entire data verification process can be realized automatically without manual participation, which improves the user experience.

In one embodiment of the present invention, after the step 102 and before the step 103, further comprising: creating a temporary table in the local database; inserting the first data result set into the temporary table In; the specific implementation manner of step 103 may include: checking the second data result set in the local database by calling and executing SQL according to the first data result set inserted in the temporary table.

In the embodiment of the present invention, a JAVA program can be used to create a temporary table. By putting data from different databases in the same database (that is, putting the first data result set obtained from the target database in the local database), and then calling and executing SQL, you can quickly compare the first The data result set and the second data result set improve execution efficiency, so that the embodiment of the present invention can realize data verification more quickly.

Besides, it can also be realized in another way. That is, after the first data result set is obtained from the target database, no temporary table is created, but the corresponding required view in the local database is firstly obtained (the required view is the same as the required view obtained from the target database same, but the same view may correspond to different data in different databases), then according to the required view, the second data result set corresponding to the first data result set in the local database can be obtained, and then Next, the second data result set obtained from the local database can be checked with the first data result set obtained from the target database to check whether the second data result set is consistent with the first data result set.

It should be noted that the first data result set and the second data result set mentioned are only for convenience of distinction, and there is no difference in order.

In one embodiment of the present invention, in order to ensure the consistency of relevant data in each database, after the step 103, it may further include: when checking that the second data result set is inconsistent with the first data result set , updating the second data result set so that the updated second data result set is consistent with the first data result set.

Since different databases may need to share related data, when it is checked that the related data stored in the two databases is inconsistent, a synchronous update is required. In the embodiment of the present invention, the first data result set in the target database is used as a standard, and then the related second data result set in the local database is updated.

It is worth noting that the mentioned target database can be any database, for example, it can also be a third-party database.

In an embodiment of the present invention, the updating of the second data result set includes: determining the newly added data in the first data result set according to the result of checking; synchronizing the added data to the the second data result set.

For example, taking the data to be checked as the employee information of the financial department as an example, assuming that the second data result set in the local database contains the basic information of two employees, "Zhang San" and "Li Si", and the information obtained from the target database The obtained first data result set contains the basic information of three employees "Zhang San", "Li Si" and "Wang Wu" (assuming that the basic information of the two employees "Zhang San" and "Li Si" is stored in the local database and are the same in the target database), it can be known by checking that the basic information of "Wang Wu" is new data in the target database, so in order to ensure data consistency, the basic information of "Wang Wu" should also be synchronized to the second data result set in the local database.

In one embodiment of the present invention, the updating of the second data result set includes: determining the modified data in the first data result set according to the checking result; The data is changed to the changed data, wherein the target data corresponds to the changed data.

Similarly, taking the data to be checked as the employee information of the financial department as an example, and the second data result set in the local database contains the basic information of two employees "Zhang San" and "Li Si", the information obtained from the target database The first data result set also contains the basic information of two employees "Zhang San" and "Li Si", but the age in the basic information of "Zhang San" in the first data result set is changed to 30 years old (assuming that other data are consistent ), then it can be known by checking that the age of 30 years old in the basic information of "Zhang San" is the changed data, so when updating the second data result set, the age of "Zhang San" in the second data result set needs to be also Change it to 30 to ensure data consistency with the first data result set.

In an embodiment of the present invention, the updating the second data result set includes: determining the deleted data in the first data result set according to the result of checking; deleting the deleted data included in the second data result set of the deleted data.

Similarly, the data to be checked is the employee information of the financial department as an example, and the second data result set in the local database contains the basic information of two employees "Zhang San" and "Li Si" in total, while the information obtained from the target database The first data result set obtained contains the basic information of an employee "Zhang San" (assuming that the basic information of the employee is consistent with the basic information of the employee in the local database), then it can be known through checking that the basic information of "Li Si" The information is the deleted data, so when updating the second data result set in the local database, the basic information of "Li Si" in the second data result set should be deleted, so as to ensure the consistency with the first data in the target database. Data consistency of the data result set.

In one embodiment of the present invention, before the step 101, it may further include: according to the preset checking cycle, periodically establish the connection information with the target database according to the preset connection information of the target database connect.

In the embodiment of the present invention, there are two ways to establish a connection with the target database. One is to use the preset connection information of the target database. The other is to obtain multiple views, and the other is to establish a connection with the target database every time a checking cycle passes, and then obtain the required views. The selection of the specific connection mode can be set by the user according to the actual needs.

Wherein, the connection information may refer to the IP of the target database, the login user name of the target database, the login password of the target database, and the like.

In one embodiment of the present invention, after the step 103, it may further include: when it is checked that the second data result set is inconsistent with the first data result set, generating feedback information for indicating data verification inconsistency ; Send the feedback information to at least one preset target address.

In order to be able to remind the relevant technical personnel, several related email addresses can be set in advance, and then when the data is inconsistent after checking, send feedback information to these email addresses to notify the relevant technical personnel of the inconsistent verification. Send specific inconsistent data to individual email addresses.

In order to be able to record, in the process of checking, corresponding logs can also be generated for subsequent search.

A data verification method provided by an embodiment of the present invention will be described in detail below. As shown in FIG. 2, the method may include the following steps:

Step 201: Establish a connection with the target database periodically according to the preset connection information of the target database according to the preset checking period.

In the embodiment of the present invention, only one connection manner is used as an example.

For example, the verification period is 48 hours, and the target database is a third-party database.

Step 202: Read the required view from the target database.

Step 203: Obtain the first data result set corresponding to the desired view from the target database.

Step 204: Create a temporary table in the local database.

In order to improve the execution efficiency of data checking in the embodiment of the present invention, the first data result set obtained from the target database is selected to be checked in the local database.

Step 205: Insert the first data result set into the temporary table.

Step 206: Check the second data result set corresponding to the first data result set in the local database by calling and executing SQL according to the first data result set inserted in the temporary table.

Step 207: When it is checked that the second data result set is inconsistent with the first data result set, update the second data result set so that the updated second data result set is consistent with the first data result set.

In the embodiment of the present invention, before the first data result set in the target database, there may be new data, changed data, and some deleted data, then when the corresponding second data result set is updated, then Operations of adding, changing, and deleting are also required, so that the second data result set in the local database can be kept consistent with the data in the first data result set after being updated.

Step 208: Generate feedback information for indicating data verification inconsistency.

Step 209: Send the feedback information to at least one preset target address.

As shown in Figure 3, an embodiment of the present invention provides a data checking device, including:

View reading unit 301, data acquiring unit 302 and data checking unit 303, wherein,

The view reading unit 301 is configured to read the required view from the target database;

The data obtaining unit 302 is configured to obtain a first data result set corresponding to the view from the target database;

The data checking unit 303 is configured to check the second data result set corresponding to the first data result set in the local database according to the first data result set.

As shown in FIG. 4, in an embodiment of the present invention, it further includes: a creation unit 401;

The creating unit 401 is configured to, after the data obtaining unit 302 obtains the first data result set corresponding to the view from the target database, and after the data checking unit 303 obtains the first data result set according to the first data The result set, before checking the second data result set corresponding to the first data result set in the local database, creating a temporary table in the local database; inserting the first data result set into the temporary table;

The data checking unit 303 is configured to check the second data result set in the local database by calling and executing SQL according to the first data result set inserted in the temporary table.

As shown in FIG. 5, in an embodiment of the present invention, it further includes: a connection unit 501;

The connection unit 501 is configured to, before the view reading unit 301 reads the required view from the target database, periodically according to the preset connection information of the target database according to the preset checking cycle , to establish a connection with the target database.

As shown in FIG. 6, in one embodiment of the present invention, it further includes: an updating unit 601;

The updating unit 601 is configured to update the second data result set when the data checking unit 303 checks that the second data result set is inconsistent with the first data result set, so that all updated data results The second data result set is consistent with the first data result set.

In an embodiment of the present invention, the update unit is configured to determine the newly added data added in the first data result set according to the checking result; and synchronize the added data to the second data result set .

In one embodiment of the present invention, the update unit is configured to determine the modified data in the first data result set according to the result of checking; change the target data in the second data result set to the modified data, wherein the target data corresponds to the modification data.

In an embodiment of the present invention, the updating unit is configured to determine the deleted data in the first data result set according to the checking result; and delete the deleted data included in the second data result set.

The information exchange and execution process among the units in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific content can refer to the description in the method embodiment of the present invention, and will not be repeated here.

To sum up, each embodiment of the present invention has at least the following beneficial effects:

1. In the embodiment of the present invention, each time the data is checked, the required view will be automatically read from the corresponding target database, and the standard first data result can be obtained from the target database through the required view In this way, according to the first data result set, the checking of the second data result set corresponding to the first data result set in the local database can be completed. The entire data verification process can be realized automatically without manual participation, which improves the user experience.

2. In the embodiment of the present invention, by placing data from different databases in the same database (that is, placing the first data result set obtained from the target database in the local database), and then executing SQL can quickly compare the first data result set and the second data result set, which improves the execution efficiency, so that the embodiment of the present invention can realize data checking more quickly.

3. In the embodiment of the present invention, when it is checked that the first data result set of the target database is inconsistent with the second data result set in the local database, by using the first data result set to update the second data result set, the The consistency between the second data result set in the local database and the first data result set in the target database is guaranteed, that is, data synchronization is realized, and different business systems using different databases can share the same data.

4. In the embodiment of the present invention, by setting the checking cycle and the connection information of the target database, each time data checking is required, the connection information can be automatically used to establish a connection with the target database, so that the following can be performed from The target database obtains the required view, and the whole process no longer requires manual intervention, which improves the efficiency of data verification.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or sequence. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a "..." does not exclude the presence of additional same elements in the process, method, article or apparatus comprising said element.

Those of ordinary skill in the art can understand that all or part of the steps to realize the above method embodiments can be completed by program instructions related hardware, and the aforementioned programs can be stored in a computer-readable storage medium. When the program is executed, the It includes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that the above descriptions are only preferred embodiments of the present invention, and are only used to illustrate the technical solution of the present invention, and are not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (10)

1. A kind of 1. verification of data method, which is characterized in that including：

   View needed for being read from target database；

   It is obtained and the corresponding first data result collection of the view from the target database；

   According to the first data result collection, the second data corresponding with the first data result collection in local data base are checked Result set.
2. 2. according to the method described in claim 1, it is characterized in that,

   After the acquisition from the target database with the corresponding first data result collection of the view, and in basis The first data result collection, check local data base in the second data result collection corresponding with the first data result collection it Before, further comprise：

   Interim table is created in the local data base；

   The first data result collection is inserted into the interim table；

   It is described according to the first data result collection, check in local data base with the first data result collection corresponding second Data result collection, including：

   According to the first data result collection being inserted into the interim table, structured query language SQL, core are performed by calling To the second data result collection in the local data base.
3. 3. according to the method described in claim 1, it is characterized in that,

   Described according to the first data result collection, check corresponding with the first data result collection the in local data base After two data result collection, further comprise：

   When checking out the second data result collection and the first data result collection is inconsistent, the second data knot is updated Fruit collects, so that the updated second data result collection is consistent with the first data result collection.
4. 4. according to the method described in claim 3, it is characterized in that,

   Update the second data result collection, including：

   According to verification as a result, determining that first data result concentrates newly-increased newly-increased data；

   The newly-increased data are synchronized to second data result to concentrate；

   And/or

   Update the second data result collection, including：

   According to verification as a result, determine the first data result concentrative change after change data；

   The target data that second data result is concentrated is changed to the change data, wherein, the target data and institute It is corresponding to state change data；

   And/or

   Update the second data result collection, including：

   According to verification as a result, determining that first data result concentrates deleted deletion data；

   It deletes second data result and concentrates the included deletion data.
5. 5. according to the method described in claim 1, it is characterized in that,

   Before view needed for reading, further comprise from target database described：

   According to the preset verification period, periodically according to the link information of the preset target database, establish with it is described Connection between target database.
6. 6. method according to any one of claims 1 to 5, which is characterized in that

   Described according to the first data result collection, check corresponding with the first data result collection the in local data base After two data result collection, further comprise：

   When checking out the second data result collection and the first data result collection is inconsistent, generation is used to indicate data core To inconsistent feedback information；

   The feedback information is sent to preset at least one destination address.
7. 7. a kind of verification of data device, which is characterized in that including：

   View reading unit, data capture unit and verification of data unit, wherein,

   The view reading unit, for view needed for the reading from target database；

   The data capture unit, for being obtained and corresponding first data result of the view from the target database Collection；

   The verification of data unit, for according to the first data result collection, checking in local data base and being counted with described first According to the corresponding second data result collection of result set.
8. 8. device according to claim 7, which is characterized in that

   Further comprise：Creating unit；

   The creating unit, it is corresponding with the view for being obtained from the target database in the data capture unit The first data result collection after, and in the verification of data unit according to the first data result collection, check local data In library before the second data result collection corresponding with the first data result collection, created in the local data base interim Table；The first data result collection is inserted into the interim table；

   The verification of data unit, for according to the first data result collection being inserted into the interim table, being held by calling Row SQL checks the second data result collection in the local data base；

   And/or

   Further comprise：Connection unit；

   The connection unit before view needed for being read from the target database in the view reading unit, is pressed According to the preset verification period, periodically according to the link information of the preset target database, establish and the number of targets According to the connection between library.
9. 9. device according to claim 7, which is characterized in that

   Further comprise：Updating unit；

   The updating unit, in the verification of data memory cell to going out the second data result collection and first data When result set is inconsistent, the second data result collection is updated, so that the updated second data result collection and described the One data result collection is consistent.
10. 10. device according to claim 9, which is characterized in that

    The updating unit, for according to verification as a result, determining that first data result concentrates newly-increased newly-increased data；It will The newly-increased data are synchronized to second data result and concentrate；And/or the updating unit, for according to verification as a result, Determine the change data after the first data result concentrative change；The target data that second data result is concentrated is changed For the change data, wherein, the target data is corresponding with the change data；And/or the updating unit, it is used for According to verification as a result, determining that first data result concentrates deleted deletion data；Delete second data result Concentrate the included deletion data.