WO2020215689A1 - Query method and apparatus for column-oriented files - Google Patents

Abstract

A query method for column-oriented files, comprising: obtaining, from a terminal, an SPL query statement input by a user (S100); determining a query range in first files of an HDFS according to the SPL query statement (S102); filtering out a second file from among the first files according to the query range (S104); converting the SPL query statement into an SQL statement according to a preset conversion rule (S106); importing the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file (S108), wherein the big data platform SQL search engine comprises Hive and/or Spark SQL; and outputting the target query file to the terminal (S110). A unified query mode is provided for users of an original log search system, the query range of an SPL statement is expanded, and convenience is provided for the querying of column-oriented data.

Description

Query method and query device for column storage files

This application requires the priority of a Chinese patent application filed with the Chinese Patent Office on April 24, 2019. The patent name is "A query method and query device for column storage files", and the application number is 201910331414.5. Incorporated in this application by reference.

Technical field

The embodiments of the present application relate to the technical field of database management, and in particular to a query method, query device, computer equipment, and readable storage medium for column storage files.

Background technique

In the current log search system, the Search Processing Language (SPL) developed by Splunk is a common search language used to query log data that has been indexed. Of course, sometimes due to disk space requirements, log data with a relatively long storage time will be stored in the form of column storage (such as parquet or optimized row columnar (orc)) in the distributed file system (Hadoop Distributed File System, HDFS) to save space. When you need to query these data, it is required to use SPL statements to query the data files in the column storage format. The inventor found that data files in the current column storage format often only support the use of Structured Query Language (SQL) as a query engine for query statements and perform data query, but do not support the use of SPL statements for query.

Therefore, this application aims to solve the problem of not supporting the SPL statement to directly query the column storage format.

Summary of the invention

In view of this, it is necessary to provide a query method, query device, computer equipment and non-volatile computer readable storage medium for column storage files, which provides a unified query mode for users of the original log search system and expands the SPL The query range of the statement provides convenience for querying column storage data.

To achieve the foregoing objective, an embodiment of the present application provides a query method for column storage files, the method including:

Obtain the SPL query sentence entered by the user from the terminal;

According to the SPL query sentence, the query range in the first file of HDFS is determined, where the first file is a column storage file, the first file is classified and stored according to a preset storage rule, and the preset storage Rules include: time sequence, application name and/or operator ID;

Filter out a second file from the first file according to the query range;

Converting the SPL query sentence into a SQL sentence according to a preset conversion rule;

Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data platform SQL search engine includes Hive and/or Spark SQL; and

Output the target query file to the terminal.

To achieve the foregoing objective, an embodiment of the present application also provides a query device for column storage files, including:

The obtaining module is used to obtain the SPL query sentence input by the user from the terminal;

The determining module is configured to determine the query range in the first file of HDFS according to the SPL query sentence, where the first file is a column storage file, and the first file is classified and stored according to a preset storage rule, The preset storage rules include: time sequence, application name and/or operator ID;

A screening module, configured to screen out a second file from the first file according to the query range;

A conversion module for converting the SPL query sentence into a SQL sentence according to a preset conversion rule;

The search module is used to import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data The platform SQL search engine includes Hive and/or Spark SQL; and

The output module is used to output the target query file to the terminal.

In order to achieve the foregoing objective, an embodiment of the present application further provides a computer device, the computer device memory, a processor, and computer-readable instructions stored in the memory and running on the processor, the computer When the readable instruction is executed by the processor, the following steps are implemented:

Obtain the SPL query sentence entered by the user from the terminal;

According to the SPL query sentence, the query range in the first file of HDFS is determined, where the first file is a column storage file, the first file is classified and stored according to a preset storage rule, and the preset storage Rules include: time sequence, application name and/or operator ID;

Filter out a second file from the first file according to the query range;

Converting the SPL query sentence into a SQL sentence according to a preset conversion rule;

Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data platform SQL search engine includes Hive and/or Spark SQL; and

Output the target query file to the terminal.

In order to achieve the above objective, the embodiments of the present application also provide a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions may Is executed by at least one processor, so that the at least one processor executes the following steps:

Obtain the SPL query sentence entered by the user from the terminal;

According to the SPL query sentence, the query range in the first file of HDFS is determined, where the first file is a column storage file, the first file is classified and stored according to a preset storage rule, and the preset storage Rules include: time sequence, application name and/or operator ID;

Filter out a second file from the first file according to the query range;

Converting the SPL query sentence into a SQL sentence according to a preset conversion rule;

Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data platform SQL search engine includes Hive and/or Spark SQL; and

Output the target query file to the terminal.

The query method, query device, computer equipment, and non-volatile computer-readable storage medium for column storage files provided by the embodiments of the present application convert SPL statements into SQL statements and determine the HDFS column storage files according to the query time range of the SPL statement The file that meets the query time range is imported into the big data platform SQL search engine, the SQL statement is executed on the big data platform SQL search engine to search for the target query file and the The target query file is output to the user terminal, providing a unified query mode for users of the original log search system, expanding the query range of SPL statements, and providing convenience for querying column storage data.

Description of the drawings

FIG. 1 is a flowchart of the steps of a method for querying stored files in the first embodiment of the application.

FIG. 2 is a schematic diagram of the hardware architecture of the query device according to the second embodiment of the application.

FIG. 3 is a schematic diagram of program modules of a storage file query system according to the third embodiment of the application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and not used to limit the application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be noted that the descriptions related to "first", "second", etc. in this application are only for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. . Therefore, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on what can be achieved by a person of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist. , Not within the scope of protection required by this application.

Example one

Referring to FIG. 1, it shows a flow chart of a method for querying stored files in the first embodiment of the present application. It can be understood that the flowchart in this method embodiment is not used to limit the order of execution of the steps. It should be noted that, in this embodiment, the query device 2 for column storage files (hereinafter referred to as query device 2) is used as the execution subject for exemplary description. details as follows:

Step S100: Obtain the SPL query sentence input by the user from the terminal.

Specifically, when the user needs to query the column storage file, the query device 2 obtains the SPL query sentence input by the user from the terminal. Wherein, the query statement includes at least: query time range and name.

Step S102: Determine the query range in the first file of HDFS according to the SPL query sentence, where the first file is a column storage file, and the first file is classified and stored according to a preset storage rule. The preset storage rules include: time sequence, application name, and operator identification information (identification, ID).

Step S104, selecting a second file from the first file according to the query range.

Specifically, when the SPL query sentence is obtained, the query time range and name of the file to be queried are identified according to the type of the SPL query sentence, and the column storage files are filtered out corresponding to the query time range and name document. It should be noted that the column storage files are stored in accordance with preset storage rules to improve query efficiency. The preset storage rules can be freely set according to the needs of users, including application name, time, etc., or Other logos are not limited here. Exemplarily, the files stored in the query device 2 are stored according to /[application name]/[year]/[month]/[day]/[hour], when it is obtained that the user uses the SPL statement to query the application name If it is syslog and the time range is from September 1st to September 3rd, 2018, the file query range is determined to be the application name syslog, and the time range is from September 1st to September 3rd, 2018. The determined query scope filters out files in the three folders /syslog/2018/9/1, /syslog/2018/9/2, and /syslog/2018/9/3.

It should be noted that when the file path stored in the query device 2 is compatible with the Hive partition, the format of each layer of the file path is [field]=[value], if the application name is represented by appname, the stored file/ The path of syslog/2018/9/1 is /appname=syslog/year=2018/month=9/day=1.

Step S106: Convert the SPL query sentence into a SQL sentence according to a preset conversion rule.

In one embodiment, the query device 2 pre-establishes a conversion mapping table for common commands of SPL sentences and common commands of SQL sentences, and stores the conversion mapping table in a database. Wherein, the conversion mapping table at least includes: common command types of SPL statements, common command types of SQL statements, and a mapping relationship between common commands of the SPL statement and common commands of the SQL statement when the view name is determined. Specifically, when the user needs to query part of the content in the database table, a view (that is, a temporary table) is created for the part of the content, and the view is named, and then the view is queried according to the input SPL query statement. When the query device 2 receives the view creation instruction input by the user, it recognizes and executes the view creation instruction to create a view. Then, when the view is created, the query device 2 obtains the view name according to the name of the view by the user, and after obtaining the view name, queries the view according to the obtained SPL query sentence. When the query device 2 receives the SPL query sentence, it identifies the command type corresponding to the SPL query sentence, and converts the SPL query sentence into a corresponding one according to the mapping relationship according to the command type and the view name. SQL statement.

Illustratively, common command types of SPL statements include: SELECT statement "Streams:[A]"; WHERE statement "[my_field]: [number]"; SELECT statement "FIELDS[A], [B]" and LIKE statement "[ A]="*some text*"". Common command types of SQL statements include: SELECT statement "SELECT*FROM[A]"; WHERE statement "SELECT*FROM[streams]WHERE[my_field]=[number]"; SELECT statement "SELECT[A],[B]" and The LIKE sentence "SELECT*FROM[streams]WHERE[A]LIKE "*some text*"".

Exemplarily, the view created by the user is named temp_1, and the content of the view includes attributes such as user name, gender, and date of birth. When the query device 2 receives the SPL query sentence input by the user: gender : [male], the query device 2 converts the SPL query statement into the SQL query statement SELECT*FROM temp_1 where gender=male according to the mapping relationship, and the mapping relationship is gender: [male] corresponds to SELECT*FROM temp_1where gender=male.

In a preferred embodiment, step S106 can be executed before step S102, and the two can also be executed in parallel, which does not affect the implementation of the embodiment of the present application.

Step S108: Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file. Wherein, the SQL search engine of the big data platform includes Hive and/or Spark SQL.

Step S110: output the target query file to the terminal.

It should be noted that if the SQL search engine of the big data platform is Hive, Hive's common data import methods include: import data from the local file system to the Hive table; import data from HDFS to the Hive table; and from other tables Query the corresponding data and import it into the Hive table; when creating the table, query the corresponding records from other tables and insert them into the created table. In this embodiment, data is imported from the HDFS into the Hive table. This importing step is an existing technology and will not be described in detail here. If the SQL search engine of the big data platform is Spark SQL, the view is created by using Spark SQL, and the converted SQL statement is used for query to output the returned result.

In a preferred embodiment, when the big data platform SQL search engine receives the file within the query range, it executes the SQL statement converted from the SPL statement, and outputs the execution result to the user terminal. The execution result is the file that the user needs to query, that is, the target query file.

In this embodiment of the application, by converting the SPL statement into the SQL statement, according to the query time range of the SPL statement, the file in the HDFS column storage file that meets the query time range is determined, and the file is imported into the big data platform SQL search engine, By executing the SQL statement on the SQL search engine of the big data platform to search for the target query file and output the target query file to the user terminal, it provides a unified query mode for users of the original log search system, and expands The query range of the SPL statement provides convenience for querying column storage data.

Example two

Please refer to FIG. 2, which shows a schematic diagram of the hardware architecture of the query device in the second embodiment of the present application. The query device 2 includes, but is not limited to, a memory 21, a processing 22, and a network interface 23 that can communicate with each other through a system bus. FIG. 2 only shows the query device 2 having components 21-23, but it should be understood that it is not It is required to implement all the illustrated components, and more or fewer components may be implemented instead.

The memory 21 includes at least one type of readable storage medium, the readable storage medium includes flash memory, hard disk, multimedia card, card type memory (for example, SD or DX memory, etc.), random access memory (RAM), static memory Random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, the memory 21 may be an internal storage unit of the query device 2, for example, a hard disk or a memory of the query device 2. In other embodiments, the memory may also be an external storage device of the query device 2, for example, a plug-in hard disk equipped on the query device 2, a smart media card (SMC), a secure digital ( Secure Digital, SD card, Flash Card, etc. Of course, the memory 21 may also include both the internal storage unit of the query device 2 and its external storage device. In this embodiment, the memory 21 is generally used to store the operating system and various application software installed in the query device 2, such as the program code of the column storage file query system 24. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

The processor 22 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 22 is generally used to control the overall operation of the query device 2. In this embodiment, the processor 22 is configured to run the program code or process data stored in the memory 21, for example, run the column storage file query system 24 and so on.

The network interface 23 may include a wireless network interface or a wired network interface, and the network interface 23 is generally used to establish a communication connection between the query device 2 and other electronic devices. For example, the network interface 23 is used to connect the query device 2 with an external terminal through a network, and establish a data transmission channel and a communication connection between the query device 2 and the external terminal. The network may be Intranet, Internet, Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G Network, Bluetooth (Bluetooth), Wi-Fi and other wireless or wired networks.

Example three

Please refer to FIG. 3, which shows a schematic diagram of program modules of the storage file query system in the third embodiment of the present application. In this embodiment, the column storage file query system 24 may include or be divided into one or more program modules. The one or more program modules are stored in a storage medium and executed by one or more processors to This application can be completed, and the query method for column storage files can be realized. The program module referred to in the embodiment of the present application refers to a series of computer-readable instruction instruction segments that can complete specific functions, and is more suitable for describing the execution process of the column storage file query system 24 in the storage medium than the program itself. The following description will specifically introduce the functions of each program module in this embodiment:

The obtaining module 201 is used to obtain the SPL query sentence input by the user from the terminal.

Specifically, when the user needs to query the column storage file, the obtaining module 201 obtains the SPL query sentence input by the user from the terminal. Wherein, the query statement includes at least: query time range and name.

The determining module 202 is configured to determine the query range in the first file of HDFS according to the SPL query sentence, where the first file is a column storage file, and the first file is classified and stored according to a preset storage rule , The preset storage rules include: time sequence, application name, and operator ID.

The screening module 203 is configured to screen out the second file from the first file according to the query range.

Specifically, when the obtaining module 201 obtains the SPL query sentence, the recognition module 208 recognizes the query time range and name of the file to be queried according to the type of the SPL query sentence, and the filtering module 203 stores the file from the column Filter out files corresponding to the query time range and name. It should be noted that the column storage files are stored in accordance with preset storage rules to improve query efficiency. The preset storage rules can be freely set according to the needs of users, including application name, time, etc., or Other logos are not limited here. Exemplarily, the files stored in the query device 2 are stored in accordance with /[application name]/[year]/[month]/[day]/[hour], when the obtaining module 201 obtains that the user uses When the SPL statement query application name is syslog, and the time range is September 1 to September 3, 2018, the identification module 208 recognizes that the file query range is the application name syslog, and the time range is September 2018 From 1st to September 3rd, the filtering module 203 filters out the three folders /syslog/2018/9/1, /syslog/2018/9/2, and /syslog/2018/9/3 according to the identified query range In the file.

It should be noted that when the file path stored in the query device 2 is compatible with the Hive partition, the format of each layer of the file path is [field]=[value], if the application name is represented by appname, the stored file/ The path of syslog/2018/9/1 is /appname=syslog/year=2018/month=9/day=1.

The conversion module 204 is configured to convert the SPL query sentence into a SQL sentence according to a preset conversion rule.

In one embodiment, the query device 2 pre-establishes a conversion mapping table for common commands of SPL sentences and common commands of SQL sentences, and stores the conversion mapping table in a database. Wherein, the conversion mapping table includes at least: common command types of SPL statements, common command types of SQL statements, and a mapping relationship between common commands of the SPL statement and common commands of the SQL statement when the view name is determined. Specifically, when the user needs to query part of the content in the table, a view (that is, a temporary table) is created for the part of the content, and the view is named, and then the view is queried according to the input SPL query statement. When the acquiring module 201 acquires the view creation instruction input by the user, the creation module 207 recognizes and executes the view creation instruction to create a view. Then, after the view is created, the obtaining module 201 obtains the view name according to the name of the view by the user, and after obtaining the view name, queries the view according to the obtained SPL query statement. When the acquisition module 201 receives the SPL query sentence, the recognition module 208 recognizes the command type corresponding to the SPL query sentence, and then, the conversion module 204 converts the command type according to the command type and the view name. The SPL query statement is converted into a corresponding SQL statement according to the mapping relationship.

Illustratively, common command types of SPL statements include: SELECT statement "Streams:[A]"; WHERE statement "[my_field]: [number]"; SELECT statement "FIELDS[A], [B]" and LIKE statement "[ A]="*some text*"". Common command types of SQL statements include: SELECT statement "SELECT*FROM[A]"; WHERE statement "SELECT*FROM[streams]WHERE[my_field]=[number]"; SELECT statement "SELECT[A],[B]" and The LIKE sentence "SELECT*FROM[streams]WHERE[A]LIKE "*some text*"".

Exemplarily, if the view created by the user is named temp_1, and the content of the view includes attributes such as user name, gender, and date of birth, when the query device 2 receives the SPL query sentence input by the user: When gender:[male], the query device 2 converts the SPL query statement into the SQL query statement SELECT*FROM temp_1 where gender=male according to the mapping relationship, and the mapping relationship is gender:[male] corresponds to SELECT* FROM temp_1where gender=male.

The search module 205 is configured to import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file. Wherein, the SQL search engine of the big data platform includes Hive and/or Spark SQL.

The output module 206 is configured to output the target query file to the terminal.

It should be noted that if the SQL search engine of the big data platform is Hive, Hive's common data import methods include: import data from the local file system to the Hive table; import data from HDFS to the Hive table; and from other tables Query the corresponding data and import it into the Hive table; when creating the table, query the corresponding records from other tables and insert them into the created table. In this embodiment, data is imported from the HDFS into the Hive table. This importing step is an existing technology and will not be described in detail here. If the SQL search engine of the big data platform is Spark SQL, the view is created by using Spark SQL, and the converted SQL statement is used for query to output the returned result.

In a preferred embodiment, when the big data platform SQL search engine receives the file within the query range, it executes the SQL statement converted from the SPL statement, and outputs the execution result to the user terminal. The execution result is the file that the user needs to query, that is, the target query file.

In this embodiment of the application, by converting the SPL statement into the SQL statement, according to the query time range of the SPL statement, the file in the HDFS column storage file that meets the query time range is determined, and the file is imported into the big data platform SQL search engine, By executing the SQL statement on the SQL search engine of the big data platform to search for the target query file and output the target query file to the user terminal, it provides a unified query mode for users of the original log search system, and expands The query range of the SPL statement provides convenience for querying column storage data.

This application also provides a computer device, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including independent servers, or more A server cluster composed of two servers), etc. The computer device in this embodiment at least includes, but is not limited to: a memory, a processor, etc. that can be communicatively connected to each other through a system bus.

This embodiment also provides a non-volatile computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory ( SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, server, App application mall, etc., on which storage There are computer-readable instructions, and the corresponding functions are realized when the program is executed by the processor. The non-volatile computer-readable storage medium of this embodiment is used to store the column storage file query system 24, and when executed by a processor, the following steps are implemented:

Obtain the SPL query sentence entered by the user from the terminal;

According to the SPL query sentence, the query range in the first file of HDFS is determined, where the first file is a column storage file, the first file is classified and stored according to a preset storage rule, and the preset storage Rules include: time sequence, application name and/or operator ID;

Filter out a second file from the first file according to the query range;

Converting the SPL query sentence into a SQL sentence according to a preset conversion rule;

Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data platform SQL search engine includes Hive and/or Spark SQL; and

Output the target query file to the terminal.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。

The above are only preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims (20)

A query method for column storage files includes the steps: Obtain the SPL query sentence entered by the user from the terminal; According to the SPL query sentence, the query range in the first file of the distributed file system is determined, where the first file is a column storage file, and the first file is classified and stored according to a preset storage rule. The preset storage rules include: time sequence, application name and/or operator ID; Filter out a second file from the first file according to the query range; Converting the SPL query sentence into a SQL sentence according to a preset conversion rule; Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data platform SQL search engine includes Hive and/or Spark SQL; and Output the target query file to the terminal. The query method according to claim 1, before the step of converting the SPL query sentence into a SQL sentence according to a preset conversion rule, the method further comprises the following step: Acquiring the view creation instruction input by the user; Identify and execute the create view instruction to create a view; and Obtain the view name input by the user. The query method according to claim 2, pre-establishing conversion mapping tables for common commands of SPL sentences and common commands of SQL sentences, and storing the conversion mapping table in a database, wherein the conversion mapping table at least includes: SPL sentences Common command types, SQL statement common command types, and the mapping relationship between the SPL statement common commands and the SQL statement common commands when the view name is determined, and the SPL query statement is converted into SQL according to a preset conversion rule The steps of the statement also include steps: Identifying the command type corresponding to the SPL query sentence; and According to the command type and the view name, the SPL query statement is converted into the SQL statement according to the mapping relationship. The query method according to claim 3, the step of filtering out the second file from the first file according to the query range, further comprising: According to the command type, identify the query time range and name of the second file; and The second file is filtered out from the column storage file. The query method of claim 1, wherein the SQL search engine of the big data platform includes Hive and/or Spark SQL. A query device for column storage files includes: The obtaining module is used to obtain the SPL query sentence input by the user from the terminal; The determining module is configured to determine the query range in the first file of the distributed file system according to the SPL query sentence, where the first file is a column storage file, and the first file is processed according to a preset storage rule Classified storage, the preset storage rules include: time sequence, application name and/or operator ID; A screening module, configured to screen out a second file from the first file according to the query range; A conversion module for converting the SPL query sentence into a SQL sentence according to a preset conversion rule; The search module is used to import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data The platform SQL search engine includes Hive and/or Spark SQL; and The output module is used to output the target query file to the terminal. The query device according to claim 6, further comprising: The acquiring module is also used to acquire the view creation instruction input by the user; The establishment module is used to identify and execute the view creation instruction to establish a view; and The obtaining module is also used to obtain the view name input by the user. The query device according to claim 7, pre-establishing conversion mapping tables for common commands of SPL sentences and common commands of SQL sentences, and storing the conversion mapping table in a database, wherein the conversion mapping table at least includes: SPL sentences The common command types, SQL statement common command types, and the mapping relationship between the SPL statement common commands and the SQL statement common commands when the view name is determined also include: An identification module for identifying the command type corresponding to the SPL query sentence; and The conversion module is configured to convert the SPL query statement into the SQL statement according to the mapping relationship according to the command type and the view name. The query device according to claim 8, further comprising: The identification module is further configured to identify the query time range and name of the second file according to the command type; and The screening module is also used to screen out the second file from the column storage file. The query device according to claim 6, wherein the SQL search engine of the big data platform includes Hive and/or Spark SQL. A computer device, the computer device memory, a processor, and computer readable instructions stored on the memory and capable of running on the processor, and when the computer readable instructions are executed by the processor, the following steps are implemented: Obtain the SPL query sentence entered by the user from the terminal; According to the SPL query sentence, the query range in the first file of the distributed file system is determined, where the first file is a column storage file, and the first file is classified and stored according to a preset storage rule. The preset storage rules include: time sequence, application name and/or operator ID; Filter out a second file from the first file according to the query range; Converting the SPL query sentence into a SQL sentence according to a preset conversion rule; Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data platform SQL search engine includes Hive and/or Spark SQL; and Output the target query file to the terminal. The computer device according to claim 11, when the computer-readable instructions are executed by the processor, the following steps are further implemented: Acquiring the view creation instruction input by the user; Identify and execute the create view instruction to create a view; and Obtain the view name input by the user. The computer device according to claim 12, pre-establishing conversion mapping tables for common commands of SPL sentences and common commands of SQL sentences, and storing the conversion mapping table in a database, wherein the conversion mapping table at least includes: SPL sentences Common command types, SQL statement common command types, and the mapping relationship between the SPL statement common commands and the SQL statement common commands when the view name is determined, the computer readable instructions are executed by the processor, and the following step: Identifying the command type corresponding to the SPL query sentence; and According to the command type and the view name, the SPL query statement is converted into the SQL statement according to the mapping relationship. The computer device according to claim 13, wherein the computer-readable instructions further implement the following steps when executed by the processor: According to the command type, identify the query time range and name of the second file; and The second file is filtered out from the column storage file. The computer device according to claim 11, the big data platform SQL search engine includes Hive and/or Spark SQL. A non-volatile computer-readable storage medium, characterized in that computer-readable instructions are stored in the non-volatile computer-readable storage medium, and the computer-readable instructions can be executed by at least one processor, So that the at least one processor executes the following steps: Obtain the SPL query sentence entered by the user from the terminal; According to the SPL query sentence, the query range in the first file of the distributed file system is determined, where the first file is a column storage file, and the first file is classified and stored according to a preset storage rule. The preset storage rules include: time sequence, application name and/or operator ID; Filter out a second file from the first file according to the query range; Converting the SPL query sentence into a SQL sentence according to a preset conversion rule; Import the second file into a big data platform SQL search engine, so that the big data platform SQL search engine executes the SQL statement to search for a target query file, wherein the big data platform SQL search engine includes Hive and/or Spark SQL; and Output the target query file to the terminal. The non-volatile computer-readable storage medium of claim 16, wherein the computer-readable instructions further implement the following steps when executed by the processor: Acquiring the view creation instruction input by the user; Identify and execute the create view instruction to create a view; and Obtain the view name input by the user. The non-volatile computer-readable storage medium according to claim 17, pre-establishing a conversion mapping table for common commands of SPL sentences and common commands of SQL sentences, and storing the conversion mapping table in a database, wherein the conversion The mapping table at least includes: common command types of SPL statements, common command types of SQL statements, and the mapping relationship between common commands of SPL statements and common commands of SQL statements when the view name is determined, and the computer-readable instructions are The processor also implements the following steps when executing: Identifying the command type corresponding to the SPL query sentence; and According to the command type and the view name, the SPL query statement is converted into the SQL statement according to the mapping relationship. The non-volatile computer-readable storage medium according to claim 18, when the computer-readable instructions are executed by the processor, the following steps are further implemented: According to the command type, identify the query time range and name of the second file; and The second file is filtered out from the column storage file. The non-volatile computer-readable storage medium according to claim 16, wherein the big data platform SQL search engine includes Hive and/or Spark SQL.

Images