[go: up one dir, main page]

CN111767279A - Data merging method and apparatus - Google Patents

Data merging method and apparatus Download PDF

Info

Publication number
CN111767279A
CN111767279A CN201910256996.5A CN201910256996A CN111767279A CN 111767279 A CN111767279 A CN 111767279A CN 201910256996 A CN201910256996 A CN 201910256996A CN 111767279 A CN111767279 A CN 111767279A
Authority
CN
China
Prior art keywords
data table
merged
initial
data
initial data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910256996.5A
Other languages
Chinese (zh)
Inventor
李超勇
马如悦
牟宇航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN201910256996.5A priority Critical patent/CN111767279A/en
Publication of CN111767279A publication Critical patent/CN111767279A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a data merging method and a data merging device, wherein the method comprises the following steps: the data table to be merged is obtained, whether the initial data table and the data table to be merged meet preset aggregation conditions or not is judged, and if the initial data table and the data table to be merged meet the preset aggregation conditions, the initial data table and the data table to be merged are merged to generate a target data table. Therefore, the generated data items are generated into the data lists in real time, and the data lists meeting the aggregation condition are combined, so that the memory is saved, the related data lists can be rapidly provided during query, and the query efficiency is improved.

Description

数据合并方法和装置Data merging method and apparatus

技术领域technical field

本申请涉及数据处理技术领域,尤其涉及一种数据合并方法和装置。The present application relates to the technical field of data processing, and in particular, to a data merging method and apparatus.

背景技术Background technique

随着互联网技术快速发展,比如广告系统每天接入大量的请求,产生海量的,如何对海量数据进行统计分析是一项非常有挑战性的任务。With the rapid development of Internet technology, for example, the advertising system accesses a large number of requests every day and generates a large amount of data. How to perform statistical analysis on the massive data is a very challenging task.

相关技术中,主要通过hadoop或者spark的批量计算来统计分析,产出统计报表,具体地,先将收集的数据写入hadoop的分布式文件系统之中,定期启动hadoop或者spark的计算任务进行离线计算,完成统计分析任务。In related technologies, the batch computing of hadoop or spark is used to perform statistical analysis and generate statistical reports. Specifically, the collected data is first written into the distributed file system of hadoop, and the computing tasks of hadoop or spark are periodically started for offline processing. Completion of statistical analysis tasks.

然而,上述方式中,有时效性上的缺失,导致无法及时获取投放广告效果、网站访问频率等关键数据,不能快速地进行业务调整、以及消耗大量的存储资源和计算资源,导致查询效率差。However, in the above methods, the lack of timeliness results in the inability to obtain key data such as advertising effects and website access frequency in a timely manner, and inability to quickly adjust services, and consume a large amount of storage and computing resources, resulting in poor query efficiency.

发明内容SUMMARY OF THE INVENTION

本申请旨在至少在一定程度上解决相关技术中的技术问题之一。The present application aims to solve one of the technical problems in the related art at least to a certain extent.

为此,本申请提出一种数据合并方法和装置,用于解决现有技术中时效性上的缺失,导致无法及时获取投放广告效果、网站访问频率等关键数据,不能快速地进行业务调整、以及消耗大量的存储资源和计算资源,导致查询效率差的技术问题,实现高效的合并数据,提升数据统计分析的性能。To this end, the present application proposes a data merging method and device, which are used to solve the lack of timeliness in the prior art, resulting in the inability to obtain key data such as advertising effects and website access frequency in a timely manner, and inability to quickly perform business adjustments, and It consumes a lot of storage resources and computing resources, which leads to technical problems of poor query efficiency, realizes efficient data consolidation, and improves the performance of data statistical analysis.

为达上述目的,本申请第一方面实施例提出了一种数据合并方法,包括:In order to achieve the above purpose, a first aspect embodiment of the present application proposes a data merging method, including:

获取待合并数据表;Get the data table to be merged;

判断初始数据表和所述待合并数据表是否满足预设的聚合条件;Judging whether the initial data table and the data table to be merged meet the preset aggregation conditions;

若判断所述初始数据表和所述待合并数据表满足预设的聚合条件,则将所述初始数据表和所述待合并数据表进行合并生成目标数据表。If it is determined that the initial data table and the data table to be merged meet the preset aggregation conditions, the initial data table and the data table to be merged are merged to generate a target data table.

本实施例的数据合并方法,通过获取待合并数据表,接着判断初始数据表和待合并数据表是否满足预设的聚合条件,若判断初始数据表和待合并数据表满足预设的聚合条件,则将初始数据表和待合并数据表进行合并生成目标数据表。由此,通过实时将产生的数据条目生成数据列表,并将满足聚合条件的各个数据列表进行合并,节约内存,以及在查询时可能快速将相关数据表提供,提高查询效率。In the data merging method of this embodiment, by acquiring the data table to be merged, then judging whether the initial data table and the data table to be merged meet the preset aggregation conditions, if it is judged that the initial data table and the data table to be merged meet the preset aggregation conditions, Then the initial data table and the data table to be merged are combined to generate the target data table. Therefore, by generating a data list from the generated data entries in real time, and merging the various data lists that meet the aggregation conditions, memory is saved, and related data tables may be quickly provided during query, thereby improving query efficiency.

为达上述目的,本申请第二方面实施例提出了一种数据合并装置,包括:In order to achieve the above purpose, a second aspect embodiment of the present application provides a data merging device, including:

第一获取模块,用于获取待合并数据表;The first acquisition module is used to acquire the data table to be merged;

判断模块,用于判断初始数据表和所述待合并数据表是否满足预设的聚合条件;a judgment module for judging whether the initial data table and the to-be-merged data table satisfy a preset aggregation condition;

合并模块,用于若判断所述初始数据表和所述待合并数据表满足预设的聚合条件,则将所述初始数据表和所述待合并数据表进行合并生成目标数据表。The merging module is configured to merge the initial data table and the data table to be merged to generate a target data table if it is determined that the initial data table and the data table to be merged meet a preset aggregation condition.

本实施例的数据合并装置,通过获取待合并数据表,接着判断初始数据表和待合并数据表是否满足预设的聚合条件,若判断初始数据表和待合并数据表满足预设的聚合条件,则将初始数据表和待合并数据表进行合并生成目标数据表。由此,通过实时将产生的数据条目生成数据列表,并将满足聚合条件的各个数据列表进行合并,节约内存,以及在查询时可能快速将相关数据表提供,提高查询效率。The data merging device of this embodiment obtains the data table to be merged, and then judges whether the initial data table and the data table to be merged meet the preset aggregation conditions, and if it is determined that the initial data table and the data table to be merged meet the preset aggregation conditions, Then the initial data table and the data table to be merged are combined to generate the target data table. Therefore, by generating a data list from the generated data entries in real time, and merging the various data lists that meet the aggregation conditions, memory is saved, and related data tables may be quickly provided during query, thereby improving query efficiency.

为达上述目的,本申请第三方面实施例提出了一种计算机设备,包括:处理器和存储器;其中,所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序,以用于实现如第一方面实施例所述的数据合并方法。In order to achieve the above purpose, an embodiment of the third aspect of the present application provides a computer device, comprising: a processor and a memory; wherein, the processor executes and executes the operation with the said processor by reading executable program codes stored in the memory. A program corresponding to the executable program code is used to implement the data merging method according to the embodiment of the first aspect.

为达上述目的,本申请第四方面实施例提出了一种非临时性计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现如第一方面实施例所述的数据合并方法。In order to achieve the above purpose, a fourth aspect embodiment of the present application provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the data as described in the first aspect embodiment is realized. Merge method.

为达上述目的,本申请第五方面实施例提出了一种计算机程序产品,当所述计算机程序产品中的指令由处理器执行时,实现如第一方面实施例所述的数据合并方法。In order to achieve the above object, the fifth aspect embodiment of the present application provides a computer program product, which implements the data merging method described in the first aspect embodiment when an instruction in the computer program product is executed by a processor.

本申请附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本申请的实践了解到。Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

附图说明Description of drawings

本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解,其中:The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

图1为本申请实施例所提供的一种数据合并方法的流程示意图;1 is a schematic flowchart of a data merging method provided by an embodiment of the present application;

图2为本申请实施例所提供的待合并数据表的示意图;2 is a schematic diagram of a data table to be merged provided by an embodiment of the present application;

图3为本申请实施例所提供的数据表实时合并存储的示意图;3 is a schematic diagram of real-time merged storage of data tables provided by an embodiment of the present application;

图4为本申请实施例所提供的数据表合并的示例图;FIG. 4 is an example diagram of data table merging provided by an embodiment of the present application;

图5为本申请实施例所提供的另一种数据合并方法的流程示意图;5 is a schematic flowchart of another data merging method provided by an embodiment of the present application;

图6为本申请实施例所提供的又一种数据合并方法的流程示意图FIG. 6 is a schematic flowchart of another data merging method provided by an embodiment of the present application

图7为本申请实施例所提供的一种数据合并装置的结构示意图;FIG. 7 is a schematic structural diagram of a data merging apparatus provided by an embodiment of the present application;

图8为本申请实施例所提供的另一种数据合并装置的结构示意图;FIG. 8 is a schematic structural diagram of another data merging apparatus provided by an embodiment of the present application;

图9为本申请实施例所提供的又一种数据合并装置的结构示意图;以及FIG. 9 is a schematic structural diagram of another data merging apparatus provided by an embodiment of the present application; and

图10为本申请实施例所提供的计算机设备的结构示意图。FIG. 10 is a schematic structural diagram of a computer device provided by an embodiment of the present application.

具体实施方式Detailed ways

下面详细描述本申请的实施例,所述实施例的示例在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的,旨在用于解释本申请,而不能理解为对本申请的限制。The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

下面参考附图描述本申请实施例的数据合并方法和装置。The data merging method and apparatus according to the embodiments of the present application will be described below with reference to the accompanying drawings.

图1为本申请实施例所提供的一种数据合并方法的流程示意图。FIG. 1 is a schematic flowchart of a data merging method provided by an embodiment of the present application.

如图1所示,该数据合并方法可以包括以下步骤:As shown in Figure 1, the data merging method may include the following steps:

步骤101,获取待合并数据表。Step 101: Obtain a data table to be merged.

本申请实施例的数据合并方法中的数据主要指的是互联网系统每时每刻都会产生海量的可以进行统计的数据,比如广告数据(比如点击、购买等),再比如用户浏览记录数据等等。The data in the data merging method of the embodiment of the present application mainly refers to that the Internet system generates massive amounts of data that can be counted at every moment, such as advertisement data (such as clicks, purchases, etc.), and user browsing record data, etc. .

基于上述背景技术的描述,现有通过批量计算分析的方式时效性上不能保证,以及需要消耗大量内存资源和计算资源,查询效率低。因此,本申请通过实时将产生的数据条目生成数据列表,并将满足聚合条件的各个数据列表进行合并,节约内存,以及在查询时可能快速将相关数据表提供,提高查询效率。具体如下:Based on the description of the above background technology, the existing methods of batch calculation and analysis cannot guarantee the timeliness, and need to consume a lot of memory resources and computing resources, and the query efficiency is low. Therefore, the present application generates a data list from the generated data entries in real time, and merges the various data lists that meet the aggregation conditions, so as to save memory, and it is possible to quickly provide relevant data tables during query, thereby improving query efficiency. details as follows:

首先获取待合并数据表,可以理解的是,待合并数据表可以是一个或者多个,可以根据预设条件选择与初始数据表合并,也可以按照预设顺序在初始数据表和一个待合并数据表进行合并后,接着与下一个待合并数据表进行合并。First, obtain the data table to be merged. It can be understood that there may be one or more data tables to be merged, which can be selected to be merged with the initial data table according to preset conditions, or can be merged between the initial data table and a data table to be merged according to a preset order. After the tables are merged, they are then merged with the next data table to be merged.

其中,获取待合并数据表的方式有很多,作为一种示例,根据预设维度个数在内存中查询获取对应的数据条目,根据数据条目生成待合并数据表;作为另一种示例,为了进一步提高合并效率,可以获取待合并数据表对应的前缀索引,根据前缀索引确定多个初始数据条目,从多个初始数据条目选取目标数据条目生成待合并数据表。Among them, there are many ways to obtain the data table to be merged. As an example, the corresponding data entry is obtained by querying the memory according to the preset number of dimensions, and the data table to be merged is generated according to the data entry; as another example, in order to further To improve the merging efficiency, the prefix index corresponding to the data table to be merged can be obtained, a plurality of initial data entries are determined according to the prefix index, and a target data entry is selected from the plurality of initial data entries to generate the data table to be merged.

为了本领域人员更加清楚上述描述,下面结合图2,以广告数据为例进行详细描述,维度可以分别为时间、广告主标号(Id)、广告地域(Country)、点击数(Clicks)和消费(Cost),比如图2表中,可以选取Time和Id两列维度作为前缀索引,在建立前缀索引的时候,比如每1024行才会在索引中有一个数据条目,所以相当于前缀索引只能定位到一个大致的范围。In order to make the above description clearer to those in the art, the following describes in detail the advertisement data in conjunction with FIG. 2 . Cost), for example, in the table in Figure 2, you can select the dimensions of Time and Id as the prefix index. When the prefix index is established, for example, there will be one data entry in the index every 1024 rows, so the equivalent prefix index can only locate to an approximate range.

比如,图2中,通过Time和Id两列维度可以确定包括时间为2013/12/31和Id为1多个初始数据条目,再从多个初始数据条目选择Country为US,从而可以确定一条目标数据条目,以及通过Time和Id两列维度可以确定包括时间为2013/12/31和Id为1对应的多个初始数据条目,再从多个初始数据条目选择Country为US,从而可以确定一条目标数据条目,通过Time和Id两列维度可以确定包括时间为2014/01/01和Id为2对应的多个初始数据条目,再从多个初始数据条目选择Country为US和UK,从而可以确定两条目标数据条目,最后根据上述目标数据条目生成待合并数据表。For example, in Figure 2, through the dimensions of Time and Id, it can be determined that the time is 2013/12/31 and the Id is 1 multiple initial data items, and then the Country is selected from the multiple initial data items as US, so that a target can be determined Data entry, and through the dimensions of Time and Id, it can be determined to include multiple initial data entries corresponding to 2013/12/31 and Id 1, and then select Country as US from multiple initial data entries, so that a target can be determined Data entry, through the two column dimensions of Time and Id, it can be determined to include multiple initial data entries corresponding to the time of 2014/01/01 and the Id of 2, and then select the Country as US and UK from the multiple initial data entries, so that the two data entries can be determined. A target data entry is generated, and finally a data table to be merged is generated according to the above target data entry.

步骤102,判断初始数据表和待合并数据表是否满足预设的聚合条件。Step 102: Determine whether the initial data table and the data table to be merged satisfy a preset aggregation condition.

步骤103,若判断初始数据表和待合并数据表满足预设的聚合条件,则将初始数据表和待合并数据表进行合并生成目标数据表。Step 103: If it is determined that the initial data table and the data table to be merged meet the preset aggregation conditions, the initial data table and the data table to be merged are merged to generate a target data table.

其中,初始数据表可以是初次导入的数据表,比如图3所示,单批次导入的数据表称为Singleton Delta,累积合并的数据表可以分为Cumulative Delta、Base Delta。其中,base是初始数据表,每导入一个批次的数据表,base就会更新,比如图3中的版本号从0开始往后增加,所以凡是从0开始计数的版本被称为base版本,不断增加导入版本的时候,数据表也在不断合并成一个更大版本,也就是可能最后合并成图3中的0-60的base版本。Among them, the initial data table can be the data table imported for the first time. For example, as shown in Figure 3, the data table imported in a single batch is called Singleton Delta, and the accumulated and merged data table can be divided into Cumulative Delta and Base Delta. Among them, base is the initial data table. Each time a batch of data tables is imported, the base will be updated. For example, the version number in Figure 3 increases from 0, so any version that counts from 0 is called the base version. When the import version is continuously added, the data table is also continuously merged into a larger version, which may finally be merged into the base version of 0-60 in Figure 3.

可以理解的是,判断初始数据表和待合并数据表是否满足预设的聚合条件的方式有很多种,作为一种可能实现的方式,获取初始数据表中各个初始维度,获取待合并数据表中各个待合并维度,根据各个初始维度和各个待合并维度是否一致判断初始数据表和待合并数据表是否满足预设的聚合条件。It can be understood that there are many ways to judge whether the initial data table and the data table to be merged meet the preset aggregation conditions. As a possible implementation method, obtain each initial dimension in the initial data table, and obtain the For each dimension to be merged, it is determined whether the initial data table and the data table to be merged meet the preset aggregation conditions according to whether each initial dimension and each dimension to be merged are consistent.

在判断初始数据表和待合并数据表满足预设的聚合条件,则将初始数据表和待合并数据表进行合并生成目标数据表,比如上述各个初始维度和各个待合并维度一致则确定初始数据表和待合并数据表满足预设的聚合条件。After judging that the initial data table and the data table to be merged meet the preset aggregation conditions, the initial data table and the data table to be merged are merged to generate the target data table. and the data table to be merged meet the preset aggregation conditions.

其中,将初始数据表和待合并数据表进行合并生成目标数据表的方式有很多种,比如求和、求最大值和求最小值等等,可以根据实际应用需要进行选择设置。Among them, there are many ways to combine the initial data table and the data table to be merged to generate the target data table, such as summation, maximum value and minimum value, etc., which can be selected and set according to actual application needs.

作为一种示例,比如图4所示的例子,将初始数据表Base和待合并数据表Delta进行求和生成目标数据表New Base,使得统计分析时,已经将数据进行合并,大大减少计算量。As an example, such as the example shown in FIG. 4 , the initial data table Base and the data table Delta to be merged are summed to generate the target data table New Base, so that the data has been merged during statistical analysis, which greatly reduces the amount of calculation.

需要说明的是,为了进一步提高合并的准确性以及后续查询的灵活性,还可以对比如上述图4中的Click、Cost维度进行判断是否满足预设交换律,在满足交换律的情况下再进行合并。It should be noted that, in order to further improve the accuracy of merging and the flexibility of subsequent queries, it is also possible to judge whether the dimensions of Click and Cost in Figure 4 above satisfy the preset exchange law, and then proceed when the exchange law is satisfied. merge.

本实施例的数据合并方法,通过获取待合并数据表,接着判断初始数据表和待合并数据表是否满足预设的聚合条件,若判断初始数据表和待合并数据表满足预设的聚合条件,则将初始数据表和待合并数据表进行合并生成目标数据表。由此,通过实时将产生的数据条目生成数据列表,并将满足聚合条件的各个数据列表进行合并,节约内存,以及在查询时可能快速将相关数据表提供,提高查询效率。In the data merging method of this embodiment, by acquiring the data table to be merged, then judging whether the initial data table and the data table to be merged meet the preset aggregation conditions, if it is judged that the initial data table and the data table to be merged meet the preset aggregation conditions, Then the initial data table and the data table to be merged are combined to generate the target data table. Therefore, by generating a data list from the generated data entries in real time, and merging the various data lists that meet the aggregation conditions, memory is saved, and related data tables may be quickly provided during query, thereby improving query efficiency.

基于上述实施例的描述,为了进一步节约内存在将初始数据表和待合并数据表进行合并生成目标数据表之后,如图5所示,还包括:Based on the description of the above embodiment, in order to further save memory, after the initial data table and the data table to be merged are merged to generate the target data table, as shown in Figure 5, it also includes:

步骤201,获取待合并数据表对应的预设删除周期。Step 201: Acquire a preset deletion period corresponding to the data table to be merged.

步骤202,在生成目标数据表的预设删除周期后将待合并数据表删除。Step 202: Delete the data table to be merged after the preset deletion period of the target data table is generated.

具体地,可以为每一个待合并数据表设置一定的删除周期比如一天、一周或者一个月等等,可以根据实际需要进行选择,在将初始数据表和待合并数据表进行合并生成目标数据表的预设删除周期后将待合并数据表删除。Specifically, a certain deletion period, such as one day, one week, or one month, can be set for each data table to be merged, which can be selected according to actual needs. After the preset deletion period, the data table to be merged will be deleted.

由此,可以节约内存,另外还可以方式用户在后续查询的时候,读取的是待合并列表这样也可以直接提供,提高查询效率,比如图3中的Cumulative Delta:61-70、61-80和61-90,可以将61-70、61-80保留一段时间,在用户直接查询61-70可以直接提供提高查询效率。In this way, memory can be saved. In addition, users can read the list to be merged in subsequent queries, which can also be provided directly to improve query efficiency. For example, Cumulative Delta in Figure 3: 61-70, 61-80 And 61-90, 61-70 and 61-80 can be reserved for a period of time, and users can directly query 61-70 to improve query efficiency.

基于上述实施例的描述,在将初始数据表和待合并数据表进行合并生成目标数据表之后,如图6所示,还包括:Based on the description of the above-mentioned embodiment, after the initial data table and the data table to be merged are combined to generate the target data table, as shown in FIG. 6 , it also includes:

步骤301,获取查询请求。Step 301, obtaining a query request.

步骤302,对查询请求进行解析,确定请求数据表。Step 302: Analyze the query request to determine the request data table.

步骤303,将请求数据表输出并显示。Step 303, outputting and displaying the request data table.

具体地,在接收到查询请求可以对查询请求进行解析,确定请求数据表比如为待合并数据表、初始数据表和目标数据表等请求数据表,可以直接将请求数据表输出并显示,提高查询效率,方便用户使用,提升用户体验。Specifically, after receiving the query request, the query request can be parsed, and the requested data table can be determined, such as the data table to be merged, the initial data table, and the target data table, etc., and the request data table can be directly output and displayed to improve the query performance. Efficiency, user-friendly, and improved user experience.

为了实现上述实施例,本申请还提出一种数据合并装置。In order to realize the above embodiments, the present application also proposes a data merging apparatus.

图7为本申请实施例所提供的一种数据合并装置的结构示意图。FIG. 7 is a schematic structural diagram of a data merging apparatus provided by an embodiment of the present application.

如图7所示,该数据合并装置可以包括:第一获取模块710、判断模块720和合并模块730。其中,As shown in FIG. 7 , the data merging apparatus may include: a first obtaining module 710 , a judging module 720 and a merging module 730 . in,

第一获取模块710,用于获取待合并数据表。The first obtaining module 710 is configured to obtain the data table to be merged.

判断模块720,用于判断初始数据表和待合并数据表是否满足预设的聚合条件。The judgment module 720 is used for judging whether the initial data table and the data table to be merged satisfy a preset aggregation condition.

合并模块730,用于若判断初始数据表和待合并数据表满足预设的聚合条件,则将初始数据表和待合并数据表进行合并生成目标数据表。The combining module 730 is configured to combine the initial data table and the data table to be combined to generate a target data table if it is determined that the initial data table and the data table to be combined meet the preset aggregation conditions.

在本申请实施例一种可能的实现方式中,第一获取模块710,具体用于:获取待合并数据表对应的前缀索引;根据前缀索引确定多个初始数据条目;从多个初始数据条目选取目标数据条目生成待合并数据表。In a possible implementation manner of the embodiment of the present application, the first obtaining module 710 is specifically configured to: obtain a prefix index corresponding to the data table to be merged; determine a plurality of initial data entries according to the prefix index; select from a plurality of initial data entries The target data entry generates the data table to be merged.

在本申请实施例一种可能的实现方式中,判断模块720,具体用于:获取初始数据表中各个初始维度;获取待合并数据表中各个待合并维度;根据各个初始维度和各个待合并维度是否一致判断初始数据表和待合并数据表是否满足预设的聚合条件。In a possible implementation manner of the embodiment of the present application, the judgment module 720 is specifically configured to: obtain each initial dimension in the initial data table; obtain each dimension to be merged in the data table to be merged; according to each initial dimension and each dimension to be merged Whether it is consistent to determine whether the initial data table and the data table to be merged meet the preset aggregation conditions.

在本申请实施例一种可能的实现方式中,如图8所示,在图7的基础上还包括:第二获取模块740和删除模块750。In a possible implementation manner of the embodiment of the present application, as shown in FIG. 8 , on the basis of FIG. 7 , it further includes: a second acquiring module 740 and a deleting module 750 .

其中,第二获取模块740,用于获取待合并数据表对应的预设删除周期。The second obtaining module 740 is configured to obtain the preset deletion period corresponding to the data table to be merged.

删除模块750,用于在生成目标数据表的预设删除周期后将待合并数据表删除。The deletion module 750 is configured to delete the data table to be merged after the preset deletion period of the target data table is generated.

在本申请实施例一种可能的实现方式中,如图9所示,在图7的基础上还包括:第三获取模块760、解析模块770和输出模块780。In a possible implementation manner of the embodiment of the present application, as shown in FIG. 9 , on the basis of FIG. 7 , it further includes: a third acquiring module 760 , a parsing module 770 and an outputting module 780 .

其中,第三获取模块760,用于获取查询请求。Wherein, the third obtaining module 760 is used to obtain the query request.

解析模块770,用于对查询请求进行解析,确定请求数据表。The parsing module 770 is configured to parse the query request and determine the request data table.

输出模块780,用于将请求数据表输出并显示。The output module 780 is used for outputting and displaying the request data table.

需要说明的是,前述对数据合并方法实施例的解释说明也适用于该实施例的数据合并装置,其实现原理类似,此处不再赘述。It should be noted that, the foregoing explanations on the data merging method embodiment are also applicable to the data merging apparatus of this embodiment, and the implementation principles thereof are similar, which will not be repeated here.

本申请实施例的数据合并装置,通过获取待合并数据表,接着判断初始数据表和待合并数据表是否满足预设的聚合条件,若判断初始数据表和待合并数据表满足预设的聚合条件,则将初始数据表和待合并数据表进行合并生成目标数据表。由此,通过实时将产生的数据条目生成数据列表,并将满足聚合条件的各个数据列表进行合并,节约内存,以及在查询时可能快速将相关数据表提供,提高查询效率。The data merging apparatus of the embodiment of the present application obtains the data table to be merged, and then judges whether the initial data table and the data table to be merged meet the preset aggregation conditions, and if it is determined that the initial data table and the data table to be merged meet the preset aggregation conditions , the initial data table and the data table to be merged are combined to generate the target data table. Therefore, by generating a data list from the generated data entries in real time, and merging the various data lists that meet the aggregation conditions, memory is saved, and related data tables may be quickly provided during query, thereby improving query efficiency.

通过为了实现上述实施例,本申请还提出一种计算机设备,包括:处理器和存储器。其中,处理器通过读取存储器中存储的可执行程序代码来运行与可执行程序代码对应的程序,以用于实现如前述实施例所述的数据合并方法。In order to realize the above embodiments, the present application also proposes a computer device, including: a processor and a memory. Wherein, the processor executes the program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the data merging method described in the foregoing embodiments.

图10为本申请实施例所提供的计算机设备的结构示意图,示出了适于用来实现本申请实施方式的示例性计算机设备90的框图。图10显示的计算机设备90仅仅是一个示例,不应对本申请实施例的功能和使用范围带来任何限制。FIG. 10 is a schematic structural diagram of a computer device provided by an embodiment of the present application, and shows a block diagram of an exemplary computer device 90 suitable for implementing the embodiment of the present application. The computer device 90 shown in FIG. 10 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

如图10所示,计算机设备90以通用计算机设备的形式表现。计算机设备90的组件可以包括但不限于:一个或者多个处理器或者处理单元906,系统存储器910,连接不同系统组件(包括系统存储器910和处理单元906)的总线908。As shown in FIG. 10, computer device 90 takes the form of a general-purpose computer device. Components of computer device 90 may include, but are not limited to, one or more processors or processing units 906 , system memory 910 , and a bus 908 connecting various system components including system memory 910 and processing unit 906 .

总线908表示几类总线结构中的一种或多种,包括存储器总线或者存储器控制器,外围总线,图形加速端口,处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说,这些体系结构包括但不限于工业标准体系结构(Industry StandardArchitecture;以下简称:ISA)总线,微通道体系结构(Micro Channel Architecture;以下简称:MAC)总线,增强型ISA总线、视频电子标准协会(Video Electronics StandardsAssociation;以下简称:VESA)局域总线以及外围组件互连(Peripheral ComponentInterconnection;以下简称:PCI)总线。Bus 908 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include, but are not limited to, Industry Standard Architecture (hereinafter referred to as: ISA) bus, Micro Channel Architecture (hereinafter referred to as: MAC) bus, enhanced ISA bus, video electronic standard Association (Video Electronics Standards Association; hereinafter referred to as: VESA) local bus and Peripheral Component Interconnection (Peripheral Component Interconnection; hereinafter referred to as: PCI) bus.

计算机设备90典型地包括多种计算机系统可读介质。这些介质可以是任何能够被计算机设备90访问的可用介质,包括易失性和非易失性介质,可移动的和不可移动的介质。Computer device 90 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 90, including both volatile and nonvolatile media, removable and non-removable media.

系统存储器910可以包括易失性存储器形式的计算机系统可读介质,例如随机存取存储器(Random Access Memory;以下简称:RAM)911和/或高速缓存存储器912。计算机设备90可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例,存储系统913可以用于读写不可移动的、非易失性磁介质(图10未显示,通常称为“硬盘驱动器”)。尽管图10中未示出,可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器,以及对可移动非易失性光盘(例如:光盘只读存储器(Compact Disc ReadOnly Memory;以下简称:CD-ROM)、数字多功能只读光盘(Digital Video Disc Read OnlyMemory;以下简称:DVD-ROM)或者其它光介质)读写的光盘驱动器。在这些情况下,每个驱动器可以通过一个或者多个数据介质接口与总线908相连。系统存储器910可以包括至少一个程序产品,该程序产品具有一组(例如至少一个)程序模块,这些程序模块被配置以执行本申请各实施例的功能。The system memory 910 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory; hereinafter referred to as: RAM) 911 and/or cache memory 912 . Computer device 90 may further include other removable/non-removable, volatile/non-volatile computer system storage media. For example only, storage system 913 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 10, commonly referred to as a "hard drive"). Although not shown in FIG. 10, disk drives for reading and writing to removable non-volatile magnetic disks (eg, "floppy disks") and removable non-volatile optical disks (eg, compact disk read only memory) may be provided. Disc ReadOnly Memory; hereinafter referred to as: CD-ROM), Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM) or other optical media) read and write optical disc drives. In these cases, each drive may be connected to bus 908 through one or more data media interfaces. System memory 910 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present application.

计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括——但不限于——电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。A computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .

计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括——但不限于——无线、电线、光缆、RF等等,或者上述的任意合适的组合。Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

可以以一种或多种程序设计语言或其组合来编写用于执行本申请操作的计算机程序代码,所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。Computer program code for performing the operations of the present application may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.

具有一组(至少一个)程序模块9140的程序/实用工具914,可以存储在例如系统存储器910中,这样的程序模块9140包括但不限于操作系统、一个或者多个应用程序、其它程序模块以及程序数据,这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块9140通常执行本申请所描述的实施例中的功能和/或方法。A program/utility 914 having a set (at least one) of program modules 9140, which may be stored, for example, in system memory 910, such program modules 9140 including, but not limited to, an operating system, one or more application programs, other program modules, and programs Data, each or some combination of these examples may include an implementation of a network environment. Program modules 9140 generally perform the functions and/or methods of the embodiments described herein.

计算机设备90也可以与一个或多个外部设备10(例如键盘、指向设备、显示器100等)通信,还可与一个或者多个使得用户能与该终端设备90交互的设备通信,和/或与使得该计算机设备90能与一个或多个其它计算设备进行通信的任何设备(例如网卡,调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口902进行。并且,计算机设备90还可以通过网络适配器900与一个或者多个网络(例如局域网(Local Area Network;以下简称:LAN),广域网(Wide Area Network;以下简称:WAN)和/或公共网络,例如因特网)通信。如图10所示,网络适配器900通过总线908与计算机设备90的其它模块通信。应当明白,尽管图10中未示出,可以结合计算机设备90使用其它硬件和/或软件模块,包括但不限于:微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。Computer device 90 may also communicate with one or more external devices 10 (eg keyboard, pointing device, display 100, etc.), may also communicate with one or more devices that enable a user to interact with the terminal device 90, and/or communicate with Any device (eg, network card, modem, etc.) that enables the computer device 90 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 902 . In addition, the computer device 90 can also communicate with one or more networks (eg, Local Area Network (hereinafter referred to as: LAN), Wide Area Network (hereinafter referred to as: WAN) and/or public networks, such as the Internet through the network adapter 900 ) ) communication. As shown in FIG. 10 , network adapter 900 communicates with other modules of computer device 90 via bus 908 . It should be understood that, although not shown in FIG. 10, other hardware and/or software modules may be used in conjunction with computer device 90, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tapes drives and data backup storage systems.

处理单元906通过运行存储在系统存储器910中的程序,从而执行各种功能应用以及数据合并,例如实现前述实施例中提及的数据合并方法。The processing unit 906 executes various functional applications and data merging by running the programs stored in the system memory 910, for example, implementing the data merging method mentioned in the foregoing embodiments.

为了实现上述实施例,本申请还提出一种非临时性计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时,实现如前述实施例所述的数据合并方法。In order to implement the above-mentioned embodiments, the present application further provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the data merging method described in the foregoing embodiments.

为了实现上述实施例,本申请还提出一种计算机程序产品,当所述计算机程序产品中的指令由处理器执行时,实现如前述实施例所述的数据合并方法。In order to implement the above-mentioned embodiments, the present application further proposes a computer program product, when the instructions in the computer program product are executed by a processor, the data merging method as described in the foregoing embodiments is implemented.

在本说明书的描述中,参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

此外,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中,“多个”的含义是至少两个,例如两个,三个等,除非另有明确具体的限定。In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.

流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为,表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分,并且本申请的优选实施方式的范围包括另外的实现,其中可以不按所示出或讨论的顺序,包括根据所涉及的功能按基本同时的方式或按相反的顺序,来执行功能,这应被本申请的实施例所属技术领域的技术人员所理解。Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

在流程图中表示或在此以其他方式描述的逻辑和/或步骤,例如,可以被认为是用于实现逻辑功能的可执行指令的定序列表,可以具体实现在任何计算机可读介质中,以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用,或结合这些指令执行系统、装置或设备而使用。就本说明书而言,“计算机可读介质”可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下:具有一个或多个布线的电连接部(电子装置),便携式计算机盘盒(磁装置),随机存取存储器(RAM),只读存储器(ROM),可擦除可编辑只读存储器(EPROM或闪速存储器),光纤装置,以及便携式光盘只读存储器(CDROM)。另外,计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质,因为可以例如通过对纸或其他介质进行光学扫描,接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序,然后将其存储在计算机存储器中。The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

应当理解,本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中,多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。如,如果用硬件来实现和在另一实施方式中一样,可用本领域公知的下列技术中的任一项或他们的组合来实现:具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路,具有合适的组合逻辑门电路的专用集成电路,可编程门阵列(PGA),现场可编程门阵列(FPGA)等。It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,该程序在执行时,包括方法实施例的步骤之一或其组合。Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

此外,在本申请各个实施例中的各功能单元可以集成在一个处理模块中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现,也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时,也可以存储在一个计算机可读取存储介质中。In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

上述提到的存储介质可以是只读存储器,磁盘或光盘等。尽管上面已经示出和描述了本申请的实施例,可以理解的是,上述实施例是示例性的,不能理解为对本申请的限制,本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (12)

1.一种数据合并方法,其特征在于,包括以下步骤:1. a data merging method, is characterized in that, comprises the following steps: 获取待合并数据表;Get the data table to be merged; 判断初始数据表和所述待合并数据表是否满足预设的聚合条件;Judging whether the initial data table and the data table to be merged meet the preset aggregation conditions; 若判断所述初始数据表和所述待合并数据表满足预设的聚合条件,则将所述初始数据表和所述待合并数据表进行合并生成目标数据表。If it is determined that the initial data table and the data table to be merged meet the preset aggregation conditions, the initial data table and the data table to be merged are merged to generate a target data table. 2.如权利要求1所述的方法,其特征在于,在所述将所述初始数据表和所述待合并数据表进行合并生成目标数据表之后,还包括:2. The method of claim 1, wherein after the initial data table and the to-be-merged data table are combined to generate a target data table, the method further comprises: 获取所述待合并数据表对应的预设删除周期;obtaining a preset deletion period corresponding to the data table to be merged; 在生成所述目标数据表的所述预设删除周期后将所述待合并数据表删除。The data table to be merged is deleted after the preset deletion period of the target data table is generated. 3.如权利要求1所述的方法,其特征在于,所述判断所述初始数据表和所述第一待合并数据表是否满足预设的聚合条件,包括:3. The method of claim 1, wherein the judging whether the initial data table and the first data table to be merged satisfy a preset aggregation condition comprises: 获取所述初始数据表中各个初始维度;obtaining each initial dimension in the initial data table; 获取所述待合并数据表中各个待合并维度;Obtain each dimension to be merged in the data table to be merged; 根据所述各个初始维度和所述各个待合并维度是否一致判断所述初始数据表和所述待合并数据表是否满足预设的聚合条件。Whether the initial data table and the to-be-merged data table satisfy a preset aggregation condition is determined according to whether the initial dimensions and the to-be-merged dimensions are consistent. 4.如权利要求1所述的方法,其特征在于,还包括:所述获取待合并数据表,包括:4. The method of claim 1, further comprising: said acquiring the data table to be merged, comprising: 获取所述待合并数据表对应的前缀索引;Obtain the prefix index corresponding to the data table to be merged; 根据所述前缀索引确定多个初始数据条目;determining a plurality of initial data entries according to the prefix index; 从所述多个初始数据条目选取目标数据条目生成所述待合并数据表。The target data entry is selected from the plurality of initial data entries to generate the data table to be merged. 5.如权利要求1所述的方法,其特征在于,在将所述初始数据表和所述待合并数据表进行合并生成目标数据表之后,还包括:5. The method according to claim 1, wherein after merging the initial data table and the data table to be merged to generate a target data table, further comprising: 获取查询请求;Get query request; 对所述查询请求进行解析,确定请求数据表;Analyze the query request to determine the request data table; 将所述请求数据表输出并显示。The request data table is output and displayed. 6.一种数据合并装置,其特征在于,包括:6. A data merging device, characterized in that, comprising: 第一获取模块,用于获取待合并数据表;The first acquisition module is used to acquire the data table to be merged; 判断模块,用于判断初始数据表和所述待合并数据表是否满足预设的聚合条件;a judgment module for judging whether the initial data table and the to-be-merged data table satisfy a preset aggregation condition; 合并模块,用于若判断所述初始数据表和所述待合并数据表满足预设的聚合条件,则将所述初始数据表和所述待合并数据表进行合并生成目标数据表。The merging module is configured to merge the initial data table and the data table to be merged to generate a target data table if it is determined that the initial data table and the data table to be merged meet a preset aggregation condition. 7.如权利要求6所述的装置,其特征在于,还包括:7. The apparatus of claim 6, further comprising: 第二获取模块,用于获取所述待合并数据表对应的预设删除周期;a second obtaining module, configured to obtain the preset deletion period corresponding to the data table to be merged; 删除模块,用于在生成所述目标数据表的所述预设删除周期后将所述待合并数据表删除。A deletion module, configured to delete the data table to be merged after the preset deletion period of the target data table is generated. 8.如权利要求6所述的装置,其特征在于,所述判断模块,具体用于:8. The device according to claim 6, wherein the judgment module is specifically used for: 获取所述初始数据表中各个初始维度;obtaining each initial dimension in the initial data table; 获取所述待合并数据表中各个待合并维度;Obtain each dimension to be merged in the data table to be merged; 根据所述各个初始维度和所述各个待合并维度是否一致判断所述初始数据表和所述待合并数据表是否满足预设的聚合条件。Whether the initial data table and the to-be-merged data table satisfy a preset aggregation condition is determined according to whether the initial dimensions and the to-be-merged dimensions are consistent. 9.如权利要求6所述的装置,其特征在于,还包括:所述第一获取模块,具体用于:9. The apparatus according to claim 6, further comprising: the first acquisition module, which is specifically configured to: 获取所述待合并数据表对应的前缀索引;Obtain the prefix index corresponding to the data table to be merged; 根据所述前缀索引确定多个初始数据条目;determining a plurality of initial data entries according to the prefix index; 从所述多个初始数据条目选取目标数据条目生成所述待合并数据表。The target data entry is selected from the plurality of initial data entries to generate the data table to be merged. 10.如权利要求6所述的装置,其特征在于,还包括:10. The apparatus of claim 6, further comprising: 第三获取模块,用于获取查询请求;The third obtaining module is used to obtain the query request; 解析模块,用于对所述查询请求进行解析,确定请求数据表;a parsing module for parsing the query request to determine the request data table; 输出模块,用于将所述请求数据表输出并显示。The output module is used for outputting and displaying the request data table. 11.一种计算机设备,其特征在于,包括处理器和存储器;11. A computer device, comprising a processor and a memory; 其中,所述处理器通过读取所述存储器中存储的可执行程序代码来运行与所述可执行程序代码对应的程序,以用于实现如权利要求1-5中任一项所述的数据合并方法。Wherein, the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to realize the data according to any one of claims 1-5 Merge method. 12.一种非临时性计算机可读存储介质,其上存储有计算机程序,其特征在于,该程序被处理器执行时实现如权利要求1-5中任一项所述的数据合并方法。12. A non-transitory computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the data merging method according to any one of claims 1-5 is implemented.
CN201910256996.5A 2019-04-01 2019-04-01 Data merging method and apparatus Pending CN111767279A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910256996.5A CN111767279A (en) 2019-04-01 2019-04-01 Data merging method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910256996.5A CN111767279A (en) 2019-04-01 2019-04-01 Data merging method and apparatus

Publications (1)

Publication Number Publication Date
CN111767279A true CN111767279A (en) 2020-10-13

Family

ID=72718128

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910256996.5A Pending CN111767279A (en) 2019-04-01 2019-04-01 Data merging method and apparatus

Country Status (1)

Country Link
CN (1) CN111767279A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114238340A (en) * 2021-12-22 2022-03-25 上海中通吉网络技术有限公司 Compact method and device for HBase table
CN115408397A (en) * 2022-09-02 2022-11-29 中国银行股份有限公司 Method, system, device and storage medium for merging data tables

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150212873A1 (en) * 2014-01-29 2015-07-30 International Business Machines Corporation Generating performance and capacity statistics
CN106547643A (en) * 2015-09-21 2017-03-29 阿里巴巴集团控股有限公司 The restoration methods and device of abnormal data
CN108268524A (en) * 2016-12-30 2018-07-10 北京国双科技有限公司 Database aggregation processing method and device
CN108446391A (en) * 2018-03-23 2018-08-24 万帮充电设备有限公司 Processing method, device, electronic equipment and the computer-readable medium of data
WO2018157680A1 (en) * 2017-03-01 2018-09-07 华为技术有限公司 Method and device for generating execution plan, and database server

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150212873A1 (en) * 2014-01-29 2015-07-30 International Business Machines Corporation Generating performance and capacity statistics
CN106547643A (en) * 2015-09-21 2017-03-29 阿里巴巴集团控股有限公司 The restoration methods and device of abnormal data
CN108268524A (en) * 2016-12-30 2018-07-10 北京国双科技有限公司 Database aggregation processing method and device
WO2018157680A1 (en) * 2017-03-01 2018-09-07 华为技术有限公司 Method and device for generating execution plan, and database server
CN108446391A (en) * 2018-03-23 2018-08-24 万帮充电设备有限公司 Processing method, device, electronic equipment and the computer-readable medium of data

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114238340A (en) * 2021-12-22 2022-03-25 上海中通吉网络技术有限公司 Compact method and device for HBase table
CN115408397A (en) * 2022-09-02 2022-11-29 中国银行股份有限公司 Method, system, device and storage medium for merging data tables

Similar Documents

Publication Publication Date Title
US11782989B1 (en) Correlating data based on user-specified search criteria
US10877987B2 (en) Correlating log data with performance measurements using a threshold value
US11119982B2 (en) Correlation of performance data and structure data from an information technology environment
US20170286499A1 (en) Query-Triggered Processing of Performance Data and Log Data from an Information Technology Environment
CN106888236B (en) Session management method and session management device
CN112100092B (en) Information caching method, device, equipment and medium
MX2013014800A (en) Recommending data enrichments.
EP2586007A2 (en) Identifying trending content items using content item histograms
CN108540508B (en) Method, device and equipment for pushing information
CN111966887B (en) Dynamic caching method and device, electronic equipment and storage medium
US20080222112A1 (en) Method and System for Document Searching and Generating to do List
US10366081B2 (en) Declarative partitioning for data collection queries
CN111767279A (en) Data merging method and apparatus
CN107291835B (en) Search term recommendation method and device
CN110096501A (en) Data search method, device, medium and electronic equipment based on label
US20210064592A1 (en) Computer storage and retrieval mechanisms using distributed probabilistic counting
US8612451B1 (en) Searching for data structures
CN110222046A (en) Processing method, device, server and the storage medium of table data
CN107480269B (en) Object display method and system, medium and computing equipment
CN119226289A (en) A corpus management method, electronic device, storage medium and program product
CN116860808A (en) Manage real-time data stream processing
CN116957354A (en) A policy evolution path analysis method, device and electronic equipment
CN110753136B (en) Domain name resolution method, device, equipment and storage medium
CN115393095A (en) Page data display method, device and equipment based on cloud computing
CN113821519A (en) Data processing method and field-driven design architecture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination