WO2017017748A1 - Computer system and sampling method - Google Patents

Abstract

Provided is a computer system that identifies work associated with a sampling candidate data group (one or more sampling candidate data items). On the basis of management information, which is information that includes information expressing a relationship between work including a data search, the work properties and the sampling rule, the computer system determines a sampling condition in accordance with a sampling rule corresponding to the properties of the identified work. Then, the computer system samples, from the sampling candidate data group, data items which meet the determined sampling condition. A sampling result is referenced at the time of the data search performed in the course of the identified work. A sampling data group, which is one or more sampling data items that include the sampled data, and/or a sampling index, which is an index of the sampling data group, is the sampling result which is referenced at the time of the data search performed in the course of the identified work.

Description

Computer system and sampling method

The present invention generally relates to a sampling technique using a computer.

If a huge amount of data group (population) is the search target (search range), it takes a long time to search. Therefore, a method of shortening the search time by considering a sampling data group composed only of data sampled in advance from the population as a search target can be considered.

For example, Patent Document 1 describes a stratified extraction mechanism provided in a database system. According to Patent Document 1, the stratification extraction mechanism defines a phrase meaning a request for stratification extraction in a query. Data from the source table is classified into different subgroups based on the stratification conditions in the query. Sampling is performed by subgroup.

US7,069,264

If a sampling data group smaller than the population is set as a search target, the work accuracy (for example, analysis accuracy) may be greatly reduced as compared to the case where the entire population is the search target. This problem is not limited to cases where the population is a database.

The computer system identifies a task associated with a sampling candidate data group that is one or more sampling candidate data. The computer system sets the sampling conditions according to the sampling rule corresponding to the specified business characteristics based on the management information, which is information including information indicating the relationship between the business including the data search and the business characteristics and the sampling rule. decide. Then, the computer system samples data that meets the determined sampling condition from the sampling candidate data group. Sampling results are referenced during data retrieval performed in the identified business. At least one of a sampling data group that is one or more sampling data including the sampled data and a sampling index that is an index of the sampling data group is used in a data search performed in the specified job. Referenced sampling result.

* Sampling results depend on sampling conditions. The sampling condition is a condition that follows a sampling rule corresponding to the characteristics of the business for which data search is to be performed. For this reason, high accuracy is expected as the accuracy of the work including the search with the sampling result as the search target.

An example of the outline | summary of embodiment is shown. A configuration example of the entire system is shown. The structural example of business management information is shown. The structural example of characteristic management information is shown. The structural example of a sampling index is shown. The structural example of rule management information is shown. The structural example of relationship management information is shown. An example of characteristic input GUI is shown. An example of the flow of a data addition process is shown. An example of the flow of a sampling process is shown. An example of the flow of a sampling condition determination process is shown. An example of the flow of maintenance processing is shown. An example of the flow of a thinning process is shown. An example of the flow of search processing is shown. The structural example of the search query by which the sampling index was designated is shown. An example of the flow of a definition process is shown. A configuration example of a definition query in which a sampling index is specified is shown.

In the following description, “database” is abbreviated as “DB” as appropriate.

In the following description, there is a case where “bbb unit” (or bbb accelerator) is used as the subject. However, these functional units perform processing determined by being executed by the processor as a memory and a communication port (network). I / F) can be used, and therefore the description may be made with the processor as the subject. The processor typically includes a microprocessor (for example, CPU (Central Processing Unit)), and further includes dedicated hardware (for example, ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array)). Good. Further, the processing disclosed with these functional units as the subject may be processing performed by a computer. In addition, some or all of these functional units may be realized by dedicated hardware. Various functional units may be installed in each computer by a program distribution server or a computer-readable storage medium. Various functional units and servers may be installed and executed on one computer, or may be installed and executed on a plurality of computers. The processor is an example of a control unit, and may include a hardware circuit that performs part or all of the processing. The program may be installed in a computer-like device from a program source. The program source may be, for example, a storage medium that can be read by a program distribution server or a computer. When the program source is a program distribution server, the program distribution server may include a processor (for example, a CPU) and a storage unit, and the storage unit may further store a distribution program and a program to be distributed. Then, the processor of the program distribution server executes the distribution program, so that the processor of the program distribution server may distribute the distribution target program to other computers. In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

In the following description, the “storage unit” may be one or more storage devices including a memory. For example, the storage unit may be at least a main storage device of a main storage device (typically a volatile memory) and an auxiliary storage device (typically a nonvolatile storage device).

In the following description, a “population” is a data group (one or more data). The “additional data group” is one or more additional data. “Additional data” is data added to the population. “Sampling” refers to extracting data from at least one of a population and an additional data group. “Sampling data” is data extracted from at least one of the population and the additional data group, that is, sampled data. The “sampling data group” is one or more sampling data.

Hereinafter, an embodiment will be described with reference to the drawings.

FIG. 1 shows an example of an outline of the embodiment.

Table 221 (table name “Table1”) is an example of a population. An additional data group that is one or more additional data 232 is repeatedly added to the table 221 (for example, regularly or irregularly). The number of additional data 232 constituting the additional data group may not be constant. The table 221 and at least the table 221 of the additional data group may be so-called big data.

In Table 221, the sampling index 223 is associated. The sampling index 223 is an index of a part of the population (sampling data group), not the entire population. Specifically, for example, the sampling index 223 may be a set of a plurality of data IDs (for example, pointers) respectively indicating a plurality of sampling data constituting the sampling data group. For example, the data composition ratio of the sampling index 223 is maintained to be the same as the data composition ratio of the table 221. The “data composition ratio” is a plurality of data ratios respectively corresponding to a plurality of value ranges, and may be a frequency distribution, for example. The “value range” may be a range composed of a minimum value and a maximum value, but may be a single value. The data ratio in the value range is the ratio of the number of target data to the total number of source data. The “total number of source data” is the number of data constituting the source data group (for example, the number of data IDs constituting the sampling index 223 or the number of data constituting the table 221 (eg, the number of records)). The “number of target data” in the value range is the number of data (or data ID of the data) including the values belonging to the value range.

The computer performs data addition processing (S101), sampling processing (S102), maintenance processing (S103), and search processing (S104).

In the data addition process (S101), for example, the following process is performed. That is, the computer adds an additional data group to the table 221. In the data addition process, a sampling process (S102) and a maintenance process (S103) are performed.

Sampling processing (S102) includes, for example, data acquisition (S102-1), condition determination (S102-2), and data sampling (S102-3).

In the data acquisition (S102-1), for example, the following processing is performed. That is, the computer acquires the business ID “BID1” corresponding to the sampling index name “S-Index1” of the sampling index 223 in the addition destination table 221 from the business management information 224 indicating the relationship between the sampling index name and the business ID. . Further, the computer acquires the characteristic details “PID1 / Y” and “PID2 / N” corresponding to the business ID “BID1” from the characteristic management information 225 indicating the relationship between the business ID and the characteristic details.

In the condition determination (S102-2), for example, the following processing is performed. That is, the computer acquires the sampling rules “rule F” and “rule G” corresponding to the characteristics “PID1 / Y” and “PID2 / N” from the rule management information 226 representing the relationship between the characteristics and the sampling rules. . Further, the computer obtains the inter-characteristic relationship “PID1 AND 特性 PID2” corresponding to the business type of the business ID “BID1” from the relationship management information 227 indicating the relationship between the business type and the inter-characteristic relationship. The relationship between characteristics is a relationship between characteristics, for example, a combination of a characteristic ID and a logical operator. The computer combines the sampling rules “Rule F” and “Rule G” corresponding to the characteristics “PID1 / Y” and “PID2 / N”, respectively, according to the inter-characteristic relationship “PID1 AND 、 PID2”. The AND rule G ”is determined. That is, each of the plurality of characteristic IDs in the inter-characteristic relationship “PID1 AND PID2” is replaced with a sampling rule corresponding to the characteristic details including the characteristic ID, and as a result, the sampling condition “rule F AND rule G” is generated. Is done. The “sampling condition” is a condition of data to be sampled.

In data sampling (S102-3), for example, the following processing is performed. That is, the computer samples data that satisfies the sampling condition “rule F AND rule G” from the additional data group.

In the maintenance process (S103), for example, the following process is performed. That is, the computer adds the data ID (for example, a pointer) of sampling data (data sampled from the additional data group) added to the table 221 to the sampling index 223 of the table 221. That is, the computer maintains the sampling index 223 associated with the table 221 to which the sampling data is added based on the result of the sampling process (S102).

This completes the data addition process (S101) including the sampling process (S102) and the maintenance process (S103).

In the search process (S104), for example, the following process is performed. That is, the computer determines whether or not the sampling index 223 is specified by a search query from a query issuer (for example, a client computer). If there is a description (for example, “QUICK S-Index1”) that means the sampling index 223 in the search query, the determination result is affirmative. If the determination result is affirmative, the computer obtains data that matches the search condition specified by the search query from only a part (sampling data group) of the table 221 through the sampling index 223 of the table 221 specified by the search query. look for. If the determination result is negative, the computer searches the entire table 221 specified by the search query for data that matches the search condition specified by the search query.

The above is the outline of this embodiment.

Various ranges can be expected as the application range of this embodiment. For example, by applying this embodiment to an analysis work for calculating the average power consumption of all households in a predetermined area, it can be expected that the analysis work will be executed in a short time. Specifically, for example, the computer according to the present embodiment is applied to a computer of an electric power company. A power company computer receives power consumption data (data representing power consumption) in a first period (for example, every minute) from sensors installed on the switchboards of each household. The computer accumulates the received power consumption data in the DB table at a second period that is the same as or different from the first period (for example, every 5 minutes). Therefore, the table is an example of the population, and the power consumption data group stored in the table is an example of the additional data group. The amount of power consumption data group depends on the number of households, but is generally large. It takes a long time to calculate the latest average power consumption using all of the power consumption data group. Therefore, a sampling index corresponding to a part of the table 221 is prepared, and when the power consumption data group is added, the computer samples and samples data from the power consumption data group according to the characteristics of the analysis work described above. The sampling index is maintained based on the collected power consumption data. Sampling indexes are referenced in searches in analytical work. Since the sampling index is updated when the power consumption data group is added, it is expected that the sampling index to be referenced is always maintained in the latest state. Further, since the sampling index is used instead of the sampling data group, the used capacity (consumed storage capacity) can be suppressed. Since the sampling index is maintained based on the data sampled according to the characteristics of the analysis work, it is expected that the work including the search using the sampling index has high accuracy.

Note that according to the present embodiment, the additional data group is an example of a sampling candidate data group (one or more sampling candidate data). As an example of the sampling candidate data group, a population as shown in Table 221 may be adopted instead of or in addition to the additional data group.

Also, the sampling index 223 is an example of the sampling result. As an example of the sampling result, instead of the sampling index 223, a sampling data group such as a part of the table 221 may be adopted. That is, the sampling index 223 may not be provided. In that case, however, the sampling data group is stored in the storage device in addition to the table 221. As a result, the used capacity of the storage device is large. This problem is even greater when table 221 is big data. For this reason, it is preferable to employ the sampling index 223 instead of the sampling data group as the sampling result as in this embodiment.

Hereinafter, this embodiment will be described in detail.

FIG. 2 shows a configuration example of the entire system.

The terminal device 201 and the external storage device 220 are connected to the computer 209.

The terminal device 201 is an example of a client system. The terminal device 201 includes an I / F (interface device) 11, an input device 202, a display device 203, a memory 12, and a CPU 207 connected thereto. The I / F 11 is connected to the computer 209 via the first communication network 208. The input device 202 may be a keyboard and a pointing device. The display device 203 may be a liquid crystal display or the like. The input device 202 and the display device 203 may be integrated like a touch panel. The memory 12 stores an OS (Operating System) 206 and a business program 205. The CPU 207 executes these programs. The business program 205 issues a query such as a search query to the computer 209 for business. The query is described in, for example, SQL (Structured Query Language).

The computer 209 is an example of a computer system. The computer 209 includes an FE-I / F (front end interface device) 210, a BE-I / F (back end interface device) 241, a memory 200, and a CPU 219 connected to them. The FE-I / F 210 and the BE-I / F 241 are examples of interface devices. The memory 200 is an example of a storage unit. The CPU 219 is an example of a processor. The terminal device 201 is connected to the FE-I / F 210 via the first communication network 208. An external storage device 220 is connected to the BE-I / F 241 via a second communication network (not shown). There may be no at least one of the first and second communication networks, or they may be the same communication network. The memory 200 stores an OS 211 and a DBMS (Database Management System) 212. The CPU 219 executes those programs.

The DBMS 212 includes a query control unit 213, a search unit 214, a definition unit 215, a data addition unit 216, a sampling unit 217, and a maintenance unit 218. The query control unit 213 receives a query and processes the query. For query processing, the query control unit 213 can call at least one of the search unit 214, the definition unit 215, the data addition unit 216, the sampling unit 217, and the maintenance unit 218 as necessary. The search unit 214 can execute a search with reference to the sampling index 223. The definition unit 215 can execute the definition of the sampling index 223. The data addition unit 216 performs addition of an additional data group, specifically, the above-described data addition process (S101). The sampling unit 217 performs sampling of data from the additional data group, specifically, the above-described sampling process (S102). The maintenance unit 218 performs maintenance of the sampling index 223, specifically, the above-described maintenance process (S103).

The external storage device 220 may be a storage system having a plurality of storage devices, for example. A plurality of storage devices may constitute one or more RAID (Redundant Array of Independent (or Inexpensive) Disks) groups. Each storage device in the RAID group is a non-volatile storage device, and may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid Disk Drive). The external storage device 220 stores a DB 242, business management information 224, characteristic management information 225, rule management information 226, relationship management information 227, DB management information 228, and characteristic GUI (Graphical User Interface) information 230. At least one of the DB 242, business management information 224, characteristic management information 225, rule management information 226, relationship management information 227, DB management information 228, and characteristic GUI information 230 may be stored in the memory 200 of the computer 209. That is, for example, the computer 209 may be an in-memory database system. Further, for example, a combination of the computer 209 and at least a part of the external storage device 220 may be an example of a computer system. For example, at least a part of the external storage device 220 may be a part of the storage unit. Further, the different types of information described above may be integrated into the same information, or may be divided into further different information. For example, the characteristic management information 225 (see FIG. 4) and the relationship management information 227 (see FIG. 7) may be integrated into one.

The DB 242 includes a table 221, a normal index 222, and a sampling index 223. The table 221 typically includes a plurality of columns and a plurality of records. At least a part of the record is an example of data constituting the table 221. One or more normal indexes 222 are associated with the table 221. One or more sampling indexes 223 are associated with at least one table 221. All normal indexes 222 associated with the table 221 refer to the entire table 221. All sampling indexes 223 associated with the table 221 refer to a part of the table 221. The normal index 222 is any index other than the sampling index 223, and is typically a general index. The sampling index 223 is an index having a data ID of sampled data. The data structure of both the normal index 222 and the sampling index 223 is not limited. That is, both the normal index 222 and the sampling index 223 may have a list structure or a B-tree structure (FIG. 5 shows a configuration example of the sampling index 223).

The business management information 224, characteristic management information 225, rule management information 226, relationship management information 227, and characteristic GUI information 230 will be described later. Information including business management information 224, characteristic management information 225, rule management information 226, and relationship management information 227 is an example of management information. The characteristic GUI information 230 may be included in the management information. The management information may further include DB management information 228.

The DB management information 228 includes information related to the DB 242. For example, the DB management information 228 is information that represents the number of data IDs (for example, the number of pointers) that the sampling index 223 has for each sampling index 223, and information that represents the relationship between the table 221 and the indexes (the normal index 222 and the sampling index 223). Index management information that is:

FIG. 3 shows a configuration example of the business management information 224.

The business management information 224 represents the relationship between the sampling index 223 and the business. Specifically, for example, the business management information 224 has entries corresponding to the sampling indexes 223, respectively. Each entry includes information such as a sampling index name 301 and a business ID 302. The sampling index name 301 represents the name of the sampling index 223 (may be another type of ID). The business ID 302 is a business ID.

FIG. 4 shows a configuration example of the characteristic management information 225.

The characteristic management information 225 represents the relationship between the business and the business characteristics. Specifically, for example, the characteristic management information 225 has an entry corresponding to each business. Each entry includes information such as a business ID 401, a characteristic ID 402, and an input value 403. A combination of the characteristic ID 402 and the input value 403 is characteristic details. A characteristic ID 402 is an ID of the characteristic. An input value 403 is a value input for the characteristic. The entry includes all the characteristic details belonging to the corresponding business. In other words, a business may be represented with one or more characteristic details. There are one or more characteristics per job.

FIG. 6 shows a configuration example of the rule management information 226.

The rule management information 226 represents the relationship between characteristics (characteristic details) and sampling rules. Specifically, for example, the rule management information 226 has an entry corresponding to each characteristic. Each entry includes information of a characteristic ID 601, an input value 602, a sampling rule 603, an external input 604, and an acquisition source 605. The sampling rule 603 represents a data sampling rule. In the sampling rule 603, the “data generation ID” is an ID that uniquely identifies a subject that generates data (for example, a sensor ID that uniquely identifies a sensor that generates sensor data), and the “sampling ratio” is This is the ratio of the number of additional sampled data (number of sampled additional data 232) to the total number of additional data (number of additional data 232 constituting the additional data group). The external input 604 represents a value input from the outside of the sampling unit 217 (DBMS 212) when the sampling condition is determined. An acquisition source 605 represents an acquisition source 605 (input source) of a value represented by the external input 604.

According to the example of FIG. 6, the characteristic details and the sampling rule 603 correspond one-to-one. According to the example of FIG. 6, for example, the following can be said. According to the sampling rule 603 corresponding to the characteristic detail “PID4 / Y”, the data is sampled so that the composition ratio of the sampling index 223 is the same as that of the table 221 with which the sampling index 223 is associated. (Sampling index 223 is maintained). Further, according to the sampling rule 603, the external input 604, and the acquisition source 605 corresponding to the characteristic details “PID5 / Y”, the priority of the additional data 232 is input by the user, and the higher the input priority, the higher the priority. Will be sampled. The “user” referred to in the present embodiment is a user of the terminal device 201, but may be an administrator of the computer 209. Further, “external storage” of the acquisition source 605 means the external storage device 220.

According to the example of FIG. 6, the characteristic details are classified into two, “Y” or “N”, for each characteristic. However, the number of characteristic details is not limited to two for at least one characteristic, and even one An integer of 3 or more may be used.

FIG. 7 shows a configuration example of the relationship management information 227.

The relationship management information 227 represents the relationship between the business and the relationship between characteristics. Specifically, for example, the relationship management information 227 has an entry corresponding to each business type. Each entry includes information such as a job type 701, a job ID 702, a characteristic ID 703, and a characteristic relationship 704. The business type 701 represents the business type 701. The business ID 702 is an ID of a business that belongs to the corresponding business type 701. A characteristic ID 703 is an ID of a characteristic belonging to the corresponding business type 701. The inter-characteristic relationship 704 represents a relationship between characteristics belonging to the corresponding business type 701.

According to the example of FIG. 7, for example, the following can be said. In other words, the characteristics that make up a business and the relationship between the characteristics differ depending on the business type. If there is one characteristic ID, the inter-characteristic relationship 704 is that one characteristic ID. On the other hand, if there are a plurality of characteristic IDs, the inter-characteristic relationship 704 is a plurality of characteristic IDs connected in series with one or more logical operators (for example, “AND” or “OR”). The inter-characteristic relationship 704 is not limited to concatenation of a plurality of characteristic IDs by a logical operator, but according to such a configuration of the inter-characteristic relationship 704, elements such as a logical operator and a characteristic ID are added, deleted, or changed It is considered that the convenience is high because the relationship 704 between the characteristics is changed by simply doing.

According to the example of FIG. 7, for example, the following can be said. In other words, when the business ID “BID2” is specified, the DBMS 212 determines from the relationship management information 227 the inter-characteristic relationship “PID1 AND PID2 AND PID5 AND PID6 corresponding to the business type“ sensor data analysis ”to which the business ID“ BID2 ”belongs. ”Is specified. Further, it is assumed that the DBMS 212 specifies the characteristic details “PID1 / Y”, “PID2 / Y”, “PID5 / Y”, and “PID6 / Y” corresponding to the business ID “BID2” from the characteristic management information 225. In this case, the sampling condition is “sampling rule corresponding to PID1 / Y sampling rule corresponding to PID2 / Y sampling rule corresponding to AND PID5 / Y sampling rule corresponding to AND PID6 / Y”. That is, each characteristic ID in the inter-characteristic relationship is replaced with a sampling rule (sampling rule 603 specified from the rule management information 226) corresponding to the characteristic details including the characteristic ID. As a result, the sampling condition is a set of sampling rules that follow the relationship between characteristics.

FIG. 8 shows an example of the characteristic input GUI.

The characteristic input GUI 800 is a GUI generated and displayed based on the characteristic GUI information 230 corresponding to the GUI. The characteristic input GUI 800 is an example of a UI (user interface) that receives an input of business characteristics. For example, the characteristic input GUI 800 is provided to the terminal device 201 by the DBMS 212 and displayed on the display device 203 of the terminal device 201. The characteristic input GUI 800 may be provided and displayed on the terminal device 201 used by the administrator of the computer 209.

According to the characteristic input GUI 800 (characteristic GUI information 230), the characteristic detail sub-UI 802 is displayed. According to the sub UI 802, for at least one characteristic, a characteristic ID, a characteristic content, an input value, and supplementary items are displayed. There is a sub UI for the input value. The sub UI for the input value may be a plurality of radio buttons respectively corresponding to a plurality of selectable input values. The supplementary matter is an example of providing the necessity of input for at least one characteristic. In other words, the supplementary items indicate whether or not the characteristics corresponding to the supplementary items are required to be input according to the business type specified by the user. The supplementary matter may further include a sentence representing a specific example (for example, for the characteristic ID “PID4”, “maintaining the composition ratio of 20% male data and 80% female data”).

The characteristic input GUI 800 may include a sub UI 801 that receives a job type from the user. When a business type is input to the sub UI 801, the DBMS 212 enables input only for characteristics that are required to be input (for example, only for a characteristic ID corresponding to the input business type), and for characteristics that do not require input (for example, (For characteristic IDs that do not correspond to the entered business type), the input may be disabled (for example, “no input” may be automatically selected). Automatic enabling / disabling of inputs may also be an example of providing input necessity for at least one characteristic.

Hereinafter, an example of processing performed in the present embodiment will be described.

FIG. 9 shows an example of the flow of data addition processing (S101).

The data adding unit 216 adds an additional data group to the table 221 (S901). The data adding unit 216 refers to the index management information in the DB management information 228, and acquires all sampling index names associated with the addition destination table 221 (S902). For each acquired sampling index name (for each sampling index 223), a sampling process (S102) and a maintenance process (S103) are performed.

Hereinafter, the sampling process (S102) and the maintenance process (S103) will be described. At that time, a sampling index 223 corresponding to one sampling index name is taken as an example. The sampling index 223 employed as an example is referred to as “target sampling index 223” in the description of FIGS.

FIG. 10 shows an example of the flow of the sampling process (S102). The sampling process (S102) may be performed, for example, by calling the sampling unit 217 by the data adding unit 216 in the data adding process (S101).

The sampling unit 217 refers to the business management information 224 and acquires the business ID 302 associated with the target sampling index name 301 (S1001).

The sampling unit 217 refers to the characteristic management information 225, and acquires characteristic details (a combination of the characteristic ID 402 and the input value 403) associated with the business ID 302 acquired in S1001 (S1002).

The sampling unit 217 executes a sampling condition determination process using the characteristic details acquired in S1002 (S1003).

The sampling unit 217 performs the following processing for each additional data included in the additional data group. That is, the sampling unit 217 determines whether or not the additional data meets the sampling condition determined in S1003 (S1011). If the determination result in S1011 is affirmative, the sampling unit 217 samples the additional data from the additional data group (S1012). The additional data sampled in S1012 is sampling data.

FIG. 11 shows an example of the flow of sampling condition determination processing (S1003).

The sampling unit 217 refers to the rule management information 226, and acquires the sampling rule 603 associated with each of the characteristic details acquired in S1002 (S1101).

The sampling unit 217 refers to the relationship management information 227, and acquires the inter-characteristic relationship 704 associated with the job type 701 to which the job ID acquired in S1001 belongs (1102).

The sampling unit 217 generates (determines) a sampling condition by combining the sampling rule 603 acquired in S1101 according to the inter-characteristic relationship 704 acquired in S1102 (S1103). Specifically, for example, each characteristic ID in the inter-characteristic relationship 704 acquired in S1102 is replaced with the sampling rule 603 associated with the characteristic details including the characteristic ID in the sampling rule 603 acquired in S1101. . The result is a sampling condition.

FIG. 12 shows an example of the flow of the maintenance process (S103). The maintenance process (S103) may be performed, for example, by calling the maintenance unit 218 by the data adding unit 216 in the data adding process (S101).

The maintenance unit 218 acquires a data ID (for example, a pointer) for each sampled data (S1201). The maintenance unit 218 adds the data ID acquired in S1201 to the target sampling index 223 (S1202). Then, the maintenance unit 218 performs a thinning process (S1203).

FIG. 13 shows an example of the flow of the thinning process (S1203).

The maintenance unit 218 determines whether the number of data IDs held by the target sampling index 223 after S1202 is greater than a certain number (S1301). The number of data IDs that the target sampling index 223 has may be specified from the DB management information 228, for example, or may be specified by scanning the target sampling index 223.

If the determination result in S1301 is negative (S1301: NO), it is not necessary to delete the data ID from the target sampling index 223, so the thinning process ends.

On the other hand, when the determination result of S1301 is affirmative (S1301: YES), the maintenance unit 218 deletes the excess number of data IDs from the target sampling index 223, thereby determining the number of data IDs of the target sampling index 223, It is made below a certain number (S1302). The maintenance unit 218 updates information indicating the number of data IDs of the target sampling index 223 in the DB management information 228 to the number of data IDs after S1302 (S1303).

As for the number of data IDs deleted in S1302, more data IDs than the excess number of data IDs may be deleted.

In S1302, there may be some variations for at least one of the deletion target candidates and the deletion conditions.

For example, the deletion target candidate (data ID that can be deleted) may be any data ID in the target sampling index 223. Further, in the target sampling index 223, the data ID added in the immediately preceding S1203 (that is, the data ID of additional data (sampling data) that meets the latest sampling condition) may be excluded from the deletion target candidates.

On the other hand, the deletion condition may be, for example, a data ID indicating data that does not conform to the latest sampling condition, or may be an old data ID. More specifically, for example, the maintenance unit 218 preferentially deletes the data ID of data that does not conform to the latest sampling condition from the target sampling index 223, and the number of data IDs of the target sampling index 223 is still a fixed number. If it exceeds, the older data ID may be deleted from the target sampling index 223 with priority. Since the data ID of data that does not conform to the latest sampling conditions is preferentially deleted, as a result, the data ID of the latest sampling data is automatically excluded from the deletion target, and the existing data that conforms to the latest sampling conditions The data ID of the data can be left preferentially.

The “latest sampling condition” is the sampling condition determined in the sampling process (S102) of the current data addition process (S101). At least one of the inter-characteristic relationship 704 and the sampling rule 603 may be changed during the data addition process. For this reason, even if the work is the same, the determined sampling conditions are not always the same. Therefore, the “latest sampling condition” may be given priority over the past sampling condition in at least one of the deletion target candidate and the deletion condition.

FIG. 14 shows an example of the flow of search processing.

For example, when a query is input from the terminal device 201 to the computer 209, the query control unit 213 analyzes the query. When the input query is a search query, the query control unit 213 calls the search unit 214, thereby starting a search process. In the search process, data that matches the search condition specified by the search query is acquired as a result set, and the search result is returned to the terminal device 201. The search result may be the acquired result set itself, or may be a result of the acquired result set being classified or aggregated.

The search unit 214 determines whether or not a hint indicating that the sampling index 223 is used is specified in the search query (S1401). The “hint” referred to here is an example of a description meaning the use of the sampling index 223. FIG. 15 shows a configuration example of a search query. According to the example, there is “QUICK-S-Index1” as a hint. In the search query, the sampling index 223 to be used may be individually specified, or only the use of the sampling index 223 may be specified. In the latter case, all the sampling indexes 223 related to the table 221 to be searched are used. The search query is a query generated by a query issuer such as the business program 205.

If the determination result in S1401 is affirmative (S1401: YES), the search unit 214 identifies the sampling index 223 from the search query (S1402), and matches the search condition specified in the search query through the specified sampling index 223. Data is acquired as a result set (S1403). Specifically, for example, for each data ID of the specified sampling index 223, the search unit 214 refers to data specified from the data ID (data in the table 221), and whether the data meets the search condition. Judge whether or not. A set of data that meets the search condition is a result set. The search unit 214 returns a result set (S1405).

If the determination result in S1401 is negative (S1401: NO), the search unit 214 uses the normal index 222 associated with the table 221 identified from the search query for data that matches the search condition specified in the search query. To obtain a result set (S1404). The search unit 214 returns a result set (S1405).

FIG. 16 shows an example of the flow of definition processing.

For example, when a query is input from the terminal device 201 to the computer 209, the query control unit 213 analyzes the query. If the input query is a definition query, the query control unit 213 calls the definition unit 215, thereby starting the definition process.

The definition unit 215 adds the table 221 name, the index name, and the index type described in the definition query to the DB management information 228 (for example, index management information) (S1601).

The definition unit 215 determines whether or not a hint indicating that the definition of the sampling index 223 is specified in the definition query (S1602). For example, according to the example of the definition query shown in FIG. 17, the hint is “ON Table1 QUICK”.

When the determination result in S1602 is affirmative (S1602: YES), the definition unit 215 provides (displays) the characteristic input GUI 800 to the terminal device 201 (S1603).

The definition unit 215 receives a business characteristic (input value) via the provided characteristic input GUI 800 (S1604), and receives a business ID and characteristic details (for example, a characteristic in which an input value other than “no input” is input). Then, it is registered in the characteristic management information 225 (S1605). The business ID 702 may be specified by a definition query, or may be an ID automatically assigned by the definition unit 215. The definition unit 215 registers the business ID and the sampling index name (index name specified by the definition query) in the business management information 224 (S1606).

Hereinafter, the embodiments will be summarized. Note that in the general description, new descriptions such as modifications of the embodiment can be added as appropriate.

The computer 209 includes a memory 200 for storing management information, which is information including information indicating the relationship between a job including data search, a property of the job, and a sampling rule, and a CPU 219 connected to the memory 200. The CPU 219 specifies (A) the business associated with the table 221 to which the additional data group is added, and (B) the sampling condition according to the sampling rule corresponding to the business characteristics specified in (A) based on the management information (C) and data that meet the sampling conditions determined in (B) are sampled from the additional data group. A sampling index 223, which is an index of sampled data, is referred to during data retrieval performed in the specified job. The sampling index 223 depends on the sampling conditions. The sampling condition is a condition that follows a sampling rule corresponding to the characteristics of the business for which data search is to be performed. For this reason, high accuracy is expected as the accuracy of the business (for example, analysis) including the search using the sampling index 223 as the search target. In addition, since sampling is performed from the additional data group, the total amount of data referred in the sampling process is typically smaller than sampling from the table 221. In the data search, since the sampling index 223 is referred to, the use capacity can be suppressed as compared with storing a copy of the sampling data group which is a part of the table 221.

The CPU 219 executes maintenance including processing for adding data indicating the data sampled in (D) and (C) to the sampling index 223. As a result, the sampling index 223 can be updated in every sampling.

The CPU 219 executes (A) to (D) when the additional data group is added to the table 221. Since sampling and maintenance are performed during the data addition process, it can be expected that the sampling index 223 is maintained in the latest state.

Table 221 and at least the table 221 of the additional data group are big data. The above-described embodiment is considered to be more effective as the size of the table 221 is larger.

In (B), the CPU 219 specifies (b1) a sampling rule corresponding to each specified business characteristic and a relationship between the specified business characteristics based on the management information, and (b2) according to the specified inter-characteristic relationship. Combine the specified sampling rules. A combination of the specified sampling loops is a sampling condition. As a result, it is possible to expect a sampling condition that is optimal for the relationship between business characteristics.

The relationship between characteristics is a combination of characteristics and logical operators. For this reason, the configuration of the inter-characteristic relationship can be changed only by changing at least one of the characteristic and the logical operator, which is highly convenient.

The specified sampling rule combination is a combination of a sampling rule corresponding to each characteristic included in the specified inter-characteristic relationship and a logical operator included in the inter-characteristic relationship. If a sampling rule is prepared for each characteristic, the sampling condition can be generated by a simple process of replacing the characteristic in the inter-characteristic relationship with the sampling rule corresponding to the characteristic according to the inter-characteristic relationship.

The CPU 219 provides a characteristic input GUI 800 that receives an input of business characteristics. The characteristic input GUI 800 provides the necessity of input corresponding to the business type specified by the user for at least one characteristic. The user can immediately specify a characteristic that requires input value input, and can input the input value of the characteristic for the specified characteristic.

At least one characteristic of the identified business is that the data composition ratio represented by the sampling index 223 is the same as the data composition ratio in the table 221. Thereby, it can be expected that the accuracy of the business including the search using the sampling index 223 is close to (for example, equivalent to) the accuracy of the business including the search referring to the entire table 221.

When the number of data IDs constituting the sampling index 223 after the maintenance of (E) and (D) exceeds a certain number, the CPU 219 starts from the sampling index 223 so that the number of data IDs in the sampling index 223 is equal to or less than the certain number. Delete the data ID. Thereby, the size of the sampling index 223 can be suppressed to a certain size or less.

In (E), the CPU 219 preferentially deletes the data ID of data that does not conform to the sampling conditions determined in (e1) and (B) from the sampling index 223, and (e2) the sampling index 223 after (e1). If the number of data IDs still exceeds a certain number, older data IDs are deleted from the sampling index 223 with higher priority. Since the data ID of data that does not conform to the latest sampling conditions is preferentially deleted, as a result, the data ID of the latest sampling data is automatically excluded from the deletion target, and the existing data that conforms to the latest sampling conditions The data ID of the data can be preferentially left in the sampling index 223.

As mentioned above, although one embodiment was described, it cannot be overemphasized that this invention can be variously changed in the range which is not limited to this embodiment and does not deviate from the summary. For example, the population is not limited to the table 221 of the database.

209 ... Calculator

Claims (15)

A storage unit for storing management information that is information including information indicating a relationship between a job including data search, a characteristic of the job, and a sampling rule;
A processor connected to the storage unit,
The processor is
(A) identifying a task associated with a sampling candidate data group that is one or more sampling candidate data;
(B) Based on the management information, determine a sampling condition in accordance with a sampling rule corresponding to the specified business characteristics;
(C) sampling data that meets the determined sampling conditions from the sampling candidate data group;
At least one of a sampling data group that is one or more sampling data including the sampled data and a sampling index that is an index of the sampling data group is used in a data search performed in the specified job. The referenced sampling result,
Computer system. The sampling candidate data group is one or more data added to a population that is one or more data,
The work associated with the sampling candidate data group is a work associated with the population to which the sampling candidate data group is added.
The computer system according to claim 1. The sampling result is the sampling index;
The sampling index is an index of a sampling data group in the population.
The computer system according to claim 2. The processor is
(D) performing maintenance including processing for adding data indicating the data sampled in (C) to the sampling index;
The computer system according to claim 3. The processor executes (A) to (D) when the sampling candidate data group is added to the population.
The computer system according to claim 4. At least the population of the population and the sampling candidate data group is big data.
The computer system according to claim 5. The processor in (B),
(B1) Based on the management information, specify a sampling rule corresponding to each of the specified business characteristics and a relationship between the specified business characteristics;
(B2) combining the specified sampling rules according to the specified relationship between characteristics,
The combination of the specified sampling loops is the sampling condition.
The computer system according to claim 1. The inter-characteristic relationship is a combination of a characteristic and a logical operator.
The computer system according to claim 7. The specified sampling rule combination is a combination of a sampling rule corresponding to each characteristic included in the specified inter-characteristic relationship and a logical operator included in the inter-characteristic relationship.
The computer system according to claim 8. The management information includes information representing the relationship between the business type and the relationship between characteristics,
The processor provides a user interface for accepting input of business characteristics;
The user interface provides, for at least one characteristic, whether or not an input is required according to a business type specified by the user;
The computer system according to claim 7. At least one characteristic of the identified work is that a data composition ratio represented by the sampling result is the same as a data composition ratio of a population that is a plurality of data.
The computer system according to claim 1. The sampling result is the sampling index;
The sampling index is an index of sampled data of the population;
The computer system according to claim 1. The processor is
(D) performing maintenance to update the sampling result based on the result of (C);
(E) When the number of data constituting the sampling result after the maintenance in (D) exceeds a certain number, the data is deleted from the sampling result so that the number of data constituting the sampling result is equal to or less than the certain number. ,
The computer system according to claim 1. The sampling candidate data group is one or more data added to a population that is one or more data,
The work associated with the sampling candidate data group is a work associated with the population to which the sampling candidate data group is added,
The processor in (E),
(E1) Data that does not conform to the sampling condition determined in (B) is preferentially deleted from the sampling result,
(E2) When the number of data constituting the sampling result after (e1) still exceeds a certain number, the older data is preferentially deleted from the sampling result.
The computer system according to claim 13. (A) identifying a task associated with a sampling candidate data group that is one or more sampling candidate data;
(B) Based on the management information, which is information including the relationship between the business including the data search, the business characteristics, and the sampling rule, the sampling condition according to the sampling rule corresponding to the specified business characteristics is determined. And
(C) sampling data that meets the determined sampling conditions from the sampling candidate data group;
At least one of a sampling data group that is one or more sampling data including the sampled data and a sampling index that is an index of the sampling data group is used in a data search performed in the specified job. The sampling result that is referenced is
Sampling method.

Images