CN120017712B - Method, equipment and storage medium for processing large data platform API data service visible domain - Google Patents
Method, equipment and storage medium for processing large data platform API data service visible domain Download PDFInfo
- Publication number
- CN120017712B CN120017712B CN202510457594.7A CN202510457594A CN120017712B CN 120017712 B CN120017712 B CN 120017712B CN 202510457594 A CN202510457594 A CN 202510457594A CN 120017712 B CN120017712 B CN 120017712B
- Authority
- CN
- China
- Prior art keywords
- data
- api
- layer
- visible domain
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, equipment and a storage medium for processing a large data platform API data service visible domain, and relates to the technical field of data processing. According to the method, access ranges of different APPs to data are controlled by defining one visible domain object, when a user layer triggers a data request, the data request is sent to an adaptation conversion layer through a gateway layer, if the adaptation conversion layer receives the data request, the adaptation conversion layer determines the value domain of the visible domain object corresponding to the triggering APP of the data request according to the APP identification associated with the data request, after a query statement is generated based on the value domain of the visible domain object and the data request, the value domain is used for determining the data access range of the triggering APP, if the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP, and therefore management and maintenance costs of API data services are reduced.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, and a storage medium for processing a large data platform API data service visible domain.
Background
In current large data platforms, data access control typically employs creating a new API (Application Programming Interface ) for each APP (Application) separately, which control method limits the user's access scope by associating APIs with specific data in a database specifying physical tables. However, with the increase of application scenes of big data services, the number of APPs and the access requirements for data have increased significantly, and the adoption of the control method described above requires frequent creation of a large number of APIs, resulting in higher maintenance costs for API data services.
Disclosure of Invention
The application mainly aims to provide a large data platform API data service visible domain processing method, equipment and storage medium, and aims to solve the technical problem of high maintenance cost of the API data service in the current data access control.
In order to achieve the above object, the present application provides a method for processing a large data platform API data service visible domain, the method comprising:
When a user layer triggers a data request, the data request is sent to an adaptation conversion layer through a gateway layer;
If the adaptation conversion layer receives the data request, the adaptation conversion layer determines a value range of a visible domain object corresponding to a trigger APP of the data request according to an APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, the query statement is sent to a calculation layer, wherein the value range is used for determining a data access range of the trigger APP;
And if the calculation layer receives the query statement, the calculation layer generates a query result according to the query statement and sends the query result to the trigger APP.
In an embodiment, the adaptation conversion layer includes an application visible domain authorizer, and before the step of sending the data request to the adaptation conversion layer by the gateway layer when the user layer triggers the data request, the adaptation conversion layer further includes:
Creating a visible domain object within the application visible domain authorizer based on a table field entered by a user;
and determining the value field of the visible field object based on the table field attribute value selected by the user by using a visible field authorizer, and establishing a mapping relation between the APP identifier and the visible field object.
In an embodiment, the gateway layer includes at least one API, the API corresponding to a database, and the step of sending, by the gateway layer, the data request to the adaptation conversion layer includes:
Based on a target API selected by a user, the gateway layer sends the data request to the adaptation conversion layer through the target API.
In an embodiment, the adaptation conversion layer further includes a task engine adapter and an API data manager, and after the step of sending the data request to the adaptation conversion layer through the gateway layer, the method further includes:
If the adaptation conversion layer receives the data request sent by the target API, the API data manager determines the type of the database corresponding to the target API according to the corresponding relation between the target API and the database;
And determining a corresponding database adapter based on the type of the database through a task engine adapter, wherein the database adapter is connected with the database and is used for executing the step of sending the query statement to a computing layer.
In an embodiment, the adaptation layer further comprises a data visible domain processing device, and the step of generating a query statement based on the value domain of the visible domain object and the data request comprises:
Generating an initial query statement by the data visible domain processing device based on a request field corresponding to the data request and a preset query field corresponding to the target API, and adding the value domain of the visible domain object as a query condition into the initial query statement to obtain the query statement.
In an embodiment, before the step of adding the value range of the visible field object as a query condition to the initial query statement to obtain the query statement, the method further includes:
analyzing the initial query statement into an abstract syntax tree through the data visible domain processing device, and determining the interpolation position of the value domain according to the position of the preset query field in the abstract syntax tree.
In an embodiment, the computing layer includes at least one database, and if the computing layer receives the query statement, the computing layer generates a query result according to the query statement, and sends the query result to the trigger APP includes:
If the calculation layer receives the query statement sent by the database adapter, the calculation layer determines a target database corresponding to the database adapter, acquires the query result from the target database after executing the query statement, and sends the query result to the adaptation conversion layer;
and sending the query result to the trigger APP through the gateway layer by the adaptation conversion layer.
In an embodiment, the adaptation conversion layer further includes a result desensitization processor, and the step of sending, by the adaptation conversion layer, the query result to the trigger APP through a gateway layer includes:
acquiring a visible field range and a field desensitization rule of the trigger APP from an application visible field authorizer through the result desensitization processor, desensitizing the query result according to the visible field range and the field desensitization rule, and then sending the desensitized query result to the gateway layer;
and sending the desensitized query result to the trigger APP in the user layer through the gateway layer.
In addition, in order to achieve the above object, the present application also proposes a large data platform API data service visible domain processing device, the device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the large data platform API data service visible domain processing method as described above.
In addition, in order to achieve the above object, the present application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, the computer program implementing the steps of the big data platform API data service visible domain processing method as described above when being executed by a processor.
The application provides a large data platform API data service visible domain processing method, which is characterized in that the control of access ranges of different APPs to data is realized by defining a visible domain object, when a user layer triggers a data request, the data request is sent to an adaptation conversion layer through a gateway layer, if the adaptation conversion layer receives the data request, the adaptation conversion layer determines the value domain of the visible domain object corresponding to the triggering APP of the data request according to the APP identifier associated with the data request, and after a query statement is generated based on the value domain of the visible domain object and the data request, the query statement is sent to a calculation layer, wherein the value domain is used for determining the data access range of the triggering APP, and if the calculation layer receives the query statement, the calculation layer generates a query result according to the query statement and sends the query result to the triggering APP.
According to the method, the adaptation conversion layer is additionally arranged between the gateway layer and the calculation layer, and the data access range of the user is limited by utilizing the value domain of the visible domain object in the adaptation conversion layer. When new APP or new data access requirements appear, the method does not need to recreate and deploy new APIs at the gateway layer, but can quickly adapt to new data access service requirements by modifying the value range of the visible domain object, so that the time and workload of the API data service are reduced, and the effect of reducing the maintenance cost of the API data service is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic diagram of a large data platform API data service visible domain processing device according to the large data platform API data service visible domain processing method in an embodiment of the present application;
FIG. 2 is a schematic flow chart of a method for processing a visible domain of an API data service of a big data platform according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of a second embodiment of a method for processing a visible domain of an API data service of a big data platform according to the present application;
FIG. 4 is a schematic diagram of a method provided in a second embodiment of a method for processing a visible domain of an API data service of a big data platform according to the present application;
FIG. 5 is a schematic flow chart of a third embodiment of a method for processing a visible domain of an API data service of a big data platform according to the present application;
FIG. 6 is a schematic structural diagram of a fourth embodiment of a method for processing a visible domain of an API data service of a big data platform according to the present application;
FIG. 7 is a schematic flow chart of a fifth embodiment of a method for processing a visible domain of an API data service of a big data platform according to the present application;
fig. 8 is a schematic diagram of a device structure of a hardware operating environment related to a method for processing a visible domain of an API data service of a big data platform according to an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the technical solution of the present application and are not intended to limit the present application.
For a better understanding of the technical solution of the present application, the following detailed description will be given with reference to the drawings and the specific embodiments.
In current large data platforms, data access control typically creates a new API for each APP's different access scope individually, by associating the API with specific data in a database specifying physical tables, thereby defining the user's access scope. However, as the number of big data service application scenarios increases, the number of APPs and the access requirements for data increase significantly. The adoption of the control method described above requires frequent creation of a large number of APIs, resulting in high maintenance costs for API data services.
In view of the above problems, the present application provides a method for processing a visible domain of an API data service of a large data platform, which implements access range control of different APPs to data by defining a visible domain object, when a user layer triggers a data request, the data request is sent to an adaptation conversion layer through a gateway layer, if the adaptation conversion layer receives the data request, the adaptation conversion layer determines a value domain of the visible domain object corresponding to the triggering APP of the data request according to an APP identifier associated with the data request, and after generating a query statement based on the value domain of the visible domain object and the data request, sends the query statement to a calculation layer, wherein the value domain is used for determining a data access range of the triggering APP, and if the calculation layer receives the query statement, the calculation layer generates a query result according to the query statement and sends the query result to the triggering APP.
According to the method, the adaptation conversion layer is additionally arranged between the gateway layer and the calculation layer, and the data access range of the user is limited by utilizing the value domain of the visible domain object in the adaptation conversion layer. When new APP or new data access requirements appear, the method does not need to recreate and deploy new APIs at the gateway layer, but can quickly adapt to new data access service requirements by modifying the value range of the visible domain object, so that the time and workload of the API data service are reduced, and the effect of reducing the maintenance cost of the API data service is achieved.
It should be noted that, the execution body of the embodiment may be a computing service device with functions of data processing, network communication and program running, for example, a tablet computer, a personal computer, or an electronic device, a large data platform API data service visible domain processing apparatus, or the like, which can implement the above functions. The present embodiment and the following embodiments will be described below by taking a large data platform API data service visible domain processing apparatus as an example.
In order to facilitate understanding of the implementation flow of the large data platform API data service visible domain processing method in the embodiment of the present application, referring to fig. 1, fig. 1 provides an architecture schematic diagram of a large data platform API data service visible domain processing device, specifically:
The large data platform API data service visible domain processing device is mainly divided into four parts, namely a user layer, a gateway layer, an adaptation conversion layer and a calculation layer.
The large data platform API data service visible domain processing device allows APPs to call APIs, each APP has a unique APP identification, and a plurality of issued library table query Restful API services can be bound. When the service request is submitted through the gateway layer, a task is submitted to the adaptation conversion layer according to the request path and the APP identification in the request header. In the adaptation conversion layer, the large data platform API data service visible domain processing device can identify the database type of the query task through the database information associated with the API, and automatically loads the required database adapter. And then dynamically generating a query statement according to the value domain of the visible domain object and the data request according to the authorized configuration of the visible domain object, and finally submitting the query statement to a calculation layer for processing.
Wherein, the user layer comprises at least one APP, the gateway layer comprises at least one API, and the API is managed by the API data gateway center. The gateway layer receives a task request from the user layer, performs authentication, security check and service validity check on the APP identification of the request, and sends a data request to the adaptation conversion layer after the verification is passed.
The adaptation conversion layer comprises an API data manager, an application visible domain authorizer, a task engine adapter, a data visible domain processing device, a log processor and a result desensitization processor. The API data manager is used for managing the API data information, and comprises an API parameter name, database information associated with the API and preset query field information associated with the API. The application visible domain authorizer is used for managing authorization of the visible domain object to the APP, limiting the data access range by accessing the value domain of the visible domain object, limiting the data receiving range of the APP, and setting the visible field range and the field desensitization rule of the query result. The task engine adapter is used for processing and adapting all task requests supported by the device and coordinating appropriate plug-ins such as a database adapter to carry out data logic processing. The data visible domain processing device utilizes ANTRL4 (Another Tool for Language Recognition, a parser generator for reading, processing, executing or translating structured text or binary files) to parse the initial query statement according to different database types, and dynamically splice the value domain in the visible domain object as the query visible domain field information to generate the physical query statement of different database engines. The log processor is used for monitoring logs of all running tasks and reporting the logs to the appointed log service center in a unified way. The result desensitization encryption processor is used for carrying out data desensitization operation on the returned query result according to the field desensitization rule.
The computing layer is an API engine resource pool composed of multiple types of databases such as MySQL, oracle, trino, postgreSQL and Neo4j (Neo 4j Graph Database), and is responsible for inquiring and acquiring API task data and providing interfaces such as inquiry job logs.
Based on this, a first embodiment of the present application provides a method for processing a large data platform API data service visible domain, referring to fig. 2, in this embodiment, the method for processing a large data platform API data service visible domain includes steps S10 to S30:
step S10, when the user layer triggers a data request, the data request is sent to the adaptation conversion layer through the gateway layer.
The gateway layer comprises at least one API, the API corresponds to the database, and when the user layer triggers the data request, the gateway layer sends the data request to the adaptation conversion layer through the target API based on the target API selected by the user.
Specifically, the APIs within the gateway layer each specify one or more databases as data sources, and the correspondence between the APIs and the databases is stored in an API data manager in the adaptation translation layer. The gateway layer forwards the data request to the adaptation conversion layer through the target API so as to ensure that the adaptation conversion layer can find a corresponding database according to the corresponding relation between the target API and the database stored in the API data manager.
Step S20, if the adaptation conversion layer receives the data request, the adaptation conversion layer determines a value range of a visible domain object corresponding to a trigger APP of the data request according to an APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, sends the query statement to a calculation layer, where the value range is used for determining a data access range of the trigger APP.
It should be noted that a visible field object refers to an object defining a data range that one or more APPs can access, and a value field defined by the visible field object is used to determine which data items are visible to the APPs. The visible domain object may be, for example, a database table, a section of a configuration file, or an object instance in a program, storing information about which data resources may be accessed by a particular user. For example, an APP of an e-commerce platform, whose visible field objects may define which merchandise information, order records, etc. the APP can view. Wherein a visible domain object may be authorized to one or more APP. Triggering an APP refers to triggering a data request in the user layer, and the APP has established a mapping relationship with a visible domain object.
Illustratively, when a trigger APP in the user layer sends a data request to the database through an API in the gateway layer, valid authentication information such as an application identifier, a user name, a user token, etc. is first sent to the gateway layer. After the triggering APP passes the authentication, the gateway layer sends a data request to the adaptation conversion layer, and the adaptation conversion layer searches a corresponding visible domain object according to the authentication information of the triggering APP and acquires the value domain of the visible domain object for use in subsequent data access control logic.
The value field of the visible field object refers to the range of the data set covered by the data access rule defined in the visible field object. The value field may be a specific database table record, a file path, a data field returned by the API, etc. For example, if a user can only view an order created by himself, the value field of his visible field object may be the user's application identifier, which is used to limit the user's data access scope.
The large data platform API data service visible domain processing device obtains the value domain of the visible domain object after determining the visible domain object corresponding to the trigger APP, and determines the data range which can be accessed by the trigger APP according to the value domain, such as order data in a specific date range, transaction data in a specific amount range and the like.
It will be appreciated that in conventional data API services, access scope control of data is primarily filtered based on the inherent attributes of the APP user, such as user name, organization structure, post role, and geographic location, and the data access scope of the user is limited by associating APIs with the inherent attributes. However, in the actual business scenario, the data visible range needs to be set according to information of other dimensions, such as inventory units of goods, pricing, preferential rates, and the like, which are not related to the attribute of the user, and at this time, a new API needs to be created and deployed to be associated with the information of the other dimensions to achieve the definition of the user data visible range. However, the embodiment proposes to define the value range of the visible range object, and further perform condition screening on the query field associated with the data request API, so as to realize the control of the visible range of the data. By dynamically adjusting the value range of the visible field object, the adjustment of service change and user requirements can be conveniently adapted without creating and deploying a new API, and the time cost and the workload cost are reduced.
Step S30, if the computing layer receives the query statement, the computing layer generates a query result according to the query statement, and sends the query result to the trigger APP.
The query statement includes specific requirements of the data that the user wishes to retrieve or analyze, such as query fields, filter criteria, ordering, etc. After receiving the query statement, the computing layer analyzes the query statement, understands the intention and the structure of the query statement, accesses the data storage of the bottom layer, such as a database, a data warehouse and the like, according to the analysis result, and executes the actual query operation. After the query is executed, the calculation layer sorts and generates a final query result, and sends the generated query result back to the APP triggering the query.
In the second embodiment of the present application, the same or similar content as in the first embodiment of the present application may be referred to the description above, and will not be repeated. On this basis, the adaptation conversion layer includes an application visible domain authorizer, please refer to fig. 3, and before step S10, the method for processing the large data platform API data service visible domain further includes steps S40 to S50:
step S40, creating a visible domain object in the application visible domain authorizer based on the table field input by the user;
illustratively, the user selects, via the management interface, a table field desired for defining the visible domain object, which is a field in a database table in the compute layer. As shown in fig. 4, it is assumed that the calculation layer stores "examination score table" in MySQL database, whose fields include "name", "city", "examination score", and "city GPD table" in Oracle database, whose fields include "quarter", "city", "GPD". When it is desired to place a range access restriction on an APP, the table field "city" may be selected as a visible domain object for defining a city range.
Step S50, determining the value domain of the visible domain object based on the table field attribute value selected by the user through the application visible domain authorizer, and establishing a mapping relation between the APP identifier and the visible domain object.
Illustratively, referring to FIG. 4, the detailed attribute value of the table field "City" is taken as the value range of the visible-range object. The identification value corresponding to the identification code of "1" is "city A", and the identification value corresponding to the identification code of "2" is "city B". By establishing the mapping relation between the APP identification and the visible domain object, the authorization of the visible domain object to the APP is realized, and when the visible domain object is authorized to the APP, the access range of the APP can be limited through the value domain of the visible domain object. For example, in fig. 4, after the mapping relationship between the visible domain object and the APP identifier is established, if the value domain identifier of the visible domain object authorized to the APP is set to "1", it indicates that the APP can only access the related data of city a.
It should be noted that, the setting form and specific value of the value range of the visible field object may be adaptively adjusted according to the actual situation, for example, fig. 4, and other value ranges may be added, where an identification value corresponding to an identification code of "3" is "$no_data", which indicates that access to any DATA is not allowed, and an identification value corresponding to an identification code of "4" is "$all_data", which indicates that access to ALL DATA is allowed.
After the mapping relation between the APP identification and the visible domain object is established, when the adaptation conversion layer receives a data request for triggering the APP, the value domain of the visible domain corresponding to the triggering APP can be determined according to the mapping relation, and the value domain is used for limiting the retrieval range of the data when the data request task is executed subsequently.
In this embodiment, by the above processing method for the visible domain of the large data platform API data service, the visible domain object can be flexibly defined according to the service requirement, and the fine control for acquiring the data range for different applications can be realized by modifying the value domain of the visible domain object, and the visible domain object can be applied to multiple APIs and databases, so that the control mode of associating a single API with a specific physical table in a specific database is not limited, the reusability of the control of the data access range is improved, and the management and maintenance costs of the API data service are reduced.
In the third embodiment of the present application, the same or similar content as the above embodiment may be referred to the above description, and will not be repeated. On this basis, the adaptation conversion layer further includes a task engine adapter and an API data manager, referring to fig. 5, after step S10, the method for processing the API data service visible domain of the big data platform further includes steps S60 to S70:
Step S60, if the adaptation conversion layer receives the data request sent by the target API, the API data manager determines, according to the correspondence between the target API and the database, the type of the database corresponding to the target API.
And step S70, determining a corresponding database adapter based on the type of the database through the task engine adapter, wherein the database adapter is connected with the database and is used for executing the step of sending the query statement to a computing layer.
It should be noted that the database adapter is a component for converting a database operation request of an application program into a format which can be understood by a specific database system, so as to realize compatibility and communication between the APP and the database. Different types of database syntax and processing logic differ, so that corresponding database adapters need to be determined according to the type of database.
The API data manager is used for managing the API data information, and comprises an API parameter name, API-associated database information and API-associated preset query fields. Optionally, a number of APIs are preset at the gateway layer, and these APIs are associated with databases in the computing layer. When triggering an APP triggers a data request to acquire data from a particular database, the data request is sent to the adaptation translation layer through a target API associated with the particular database. The adaptation conversion layer can determine the database corresponding to the target API through the corresponding relation between the target API and the database stored in the API data manager, then determine the corresponding database adapter through the task engine adapter, and send the query statement corresponding to the data request to the corresponding database in the calculation layer through the database adapter.
In this embodiment, the adaptation translation layer incorporates a task engine adapter and an API data manager. When the adaptation conversion layer receives a data request sent by the target API, the API data manager accurately determines the type of the database corresponding to the target API according to the corresponding relation between the target API and the database. The task engine adapter then matches the corresponding database adapter according to the database type, which is connected to the database and is responsible for delivering the query statement to the compute layer. The database adapter is used as a key component, can convert database operation requests of application programs into formats which can be identified by a specific database system, and ensures compatibility and smooth communication between the APP and databases of different types. By presetting an API at the gateway layer and associating with the computation layer database, when an APP triggers a data request to acquire data from a specific database, a target API sends the request to the adaptation conversion layer. The adaptation conversion layer determines a database and an adapter corresponding to the target API by using the corresponding relation stored in the API data manager, and then the database adapter sends the query statement to the database corresponding to the calculation layer, so that the data request processing is more efficient and accurate, different database environments can be better adapted, and the accuracy and the efficiency of data acquisition are improved.
Based on the above embodiment of the present application, in a fourth embodiment of the present application, the adaptation conversion layer further includes a data visible domain processing device, and the step of generating the query statement based on the value domain of the visible domain object and the data request in step S20 includes that the data visible domain processing device generates an initial query statement based on a preset query field corresponding to the target API and a request field corresponding to the data request, and inserts the value domain of the visible domain object into the initial query statement as a query condition to obtain the query statement.
In the conventional data access control method, the API is generally associated with specific data of a specified physical table in the database, so as to limit the access range of the user. However, in this embodiment, the API is only associated with the database, and a preset query field may be set in advance for the API, and when the API receives a data request, an initial query statement is formed according to a request field corresponding to the request data and the preset query field, and then the access range of the user is further limited according to the value range of the visible domain object.
Illustratively, a preset query field "SELECT%field%from%table%where% CONDITIONS%", WHERE "%" represents a placeholder, is set for each API. When the adaptation conversion layer receives a data request, a request field appointed by a user is extracted from the data request, and the extracted request field is filled in a placeholder position in a preset query field. As shown in fig. 2, assuming that the APP triggers a data request to query all test score information FROM the MySQL database, the request field "test score table" is filled into the placeholder position of the preset query field in the test score API, an initial query statement "SELECT FROM test score table" is generated, and then the access range of the user is further limited according to the value range "city a" of the visible domain object, and the query statement "SELECT FROM test score table WHERE city= 'city a'" is generated.
For better understanding of the solution presented in this example, this example is further described in connection with a specific application scenario.
Referring to fig. 6, basic information of a visible field object, such as an object name, an object code, etc., is defined, and then a value field of the visible field object, such as a date range, an amount range, a sequence range, or a specific dictionary value, such as a city name, a department name, an academy, etc., is defined. Some APP calls multiple APIs including "order API", "customer API", "space coordinates API". The method comprises the steps of storing a database name of Mysql-x, a table name of "order_t", a field name of "order_city", a database name of "MySQL-x", a table name of "cure_t", a field name of "c_city", a database name of "PostGIS-x", a table name of "city_ gis _t" and a field name of "g_cc. In the conventional data access range control method, when the APP calls the APIs to query city data, if the city range that each API can access is inconsistent, each API designation association needs to be designated to a specific city in the table, so as to define the data access range of the user side. However, by the method for processing the visible domain of the large data platform API data service provided by the embodiment, a mapping relation can be established between the defined visible domain object and the APP identifier, and the value domain of the visible domain object corresponding to the APP is obtained through the mapping relation, so that the urban range which the APP can access is limited. One visible domain object can be mapped with a plurality of APPs, can control the access range of a plurality of APIs and is applied to a plurality of databases. Wherein MySQL, oracle, trino, postgreSQL and Neo4j are five different types of databases, in other embodiments, the database engine includes, but is not limited to, the five types described above.
It can be appreciated that, because a mapping relationship is established between the visible domain object and the APP identifier, when the query authority of the APP changes, that is, the data access range of the APP changes, the related API request processing logic can be automatically updated by modifying the value domain of the visible domain object or specifying the specific table field attribute value authorized for the APP in the value domain of the visible domain object, without requiring manual modification of each API.
Optionally, the value field of the visible field object is used as a query condition and is added into an initial query statement, before the step of obtaining the query statement, the data visible field processing device analyzes the initial query statement into an abstract syntax tree, and the adding and inserting position of the value field is determined according to the position of a preset query field in the abstract syntax tree.
It should be noted that the abstract syntax tree is a tree representation of an abstract syntax structure of the source code, and each node in the abstract syntax tree represents a structure in the source code, such as an expression, a statement, and the like.
Illustratively, based on ANTRL4, different types of query statements are parsed into abstract syntax trees by lexical rule files, such as the lexer. G4 file, and syntax rule files, such as the Parser. G4 file. Then, traversing the abstract syntax tree, accessing custom Visitor and Listener classes of nodes in the abstract syntax tree, and identifying the semantics in the query statement and preset query fields such as SELECT, FROM, JOIN, WHERE. After the position of the preset query field is determined, the value domain of the visible domain object is subjected to assignment and splicing, so that the filtering condition is dynamically added in the query statement, and a final query statement is generated. Wherein ANTRL is a grammar generator tool, visitor class contains methods to access each node in the abstract syntax tree, and then all nodes in the abstract syntax tree can be accessed recursively by calling the visit method of Visitor and passing into the root node of the abstract syntax tree. The Listener class contains event handling methods for each node in the abstract syntax tree upon entry and exit, which the ANTLR4 can automatically invoke to traverse the abstract syntax tree.
By traversing the nodes of the abstract syntax tree, according to the type and the position of the query field in the nodes, the condition of the value domain is determined where to insert, and the filtering field addition of the query statement can be dynamically realized, so that the modification and the expansion of the query statement are more visual and easier to manage.
In the fifth embodiment of the present application, the same or similar contents as those of the above embodiment can be referred to the above description, and the description thereof will be omitted. On this basis, the computing layer includes at least one database, referring to fig. 7, after step S30, the method for processing the API data service visible domain of the big data platform further includes steps S80 to S90:
Step S80, if the computing layer receives the query statement sent by the database adapter, the computing layer determines a target database corresponding to the database adapter, and after executing the query statement, obtains the query result from the target database, and sends the query result to the adaptation conversion layer.
It should be noted that, different database adapters correspond to different types of databases, and the database adapters will convert the query statement into a format that can be understood by a specific database, and then send the query statement to the computing layer. The computing layer firstly receives the query statement sent by the database adapter, then executes the query statement to obtain a query result from the corresponding target database, and then sends the query result back to the adaptation conversion layer. The target database is a database corresponding to the database adapter.
Step S90, sending, by the adaptation conversion layer, the query result to the trigger APP through a gateway layer.
Optionally, the adaptive conversion layer further includes a result desensitizing processor, and step S90 includes steps S91 to S92:
Step S91, obtaining a visible field range and a field desensitization rule of the trigger APP from an application visible field authorizer through the result desensitization processor, desensitizing the query result according to the visible field range and the field desensitization rule, and then sending the desensitized query result to the gateway layer.
It should be noted that the result desensitizing processor is a component or module for processing data desensitization, and is used for processing sensitive data according to a preset field desensitizing rule to protect data privacy, for example, replacing personal address information with anonymous numbers.
The application visible domain authorizer stores the visible field range and the field desensitization rule of each APP, and when the query result is returned to the adaptation conversion layer, the result desensitization processor firstly acquires the visible field range and the field desensitization rule of the triggering APP from the application visible domain authorizer. The field desensitization rule may be set according to the data type of the query result, for example, coding desensitization is performed on numeric data, encryption desensitization is performed on text data, and the like.
Step S92, sending, by the gateway layer, the desensitized query result to the trigger APP in the user layer.
In this embodiment, the database adapter can convert the query statement into a format that can be understood by a specific database, so as to ensure that databases of different types can correctly parse and execute the query operation. Further, the result desensitization processor in the adaptation conversion layer can acquire the visible field range and the field desensitization rule of the trigger APP from the application visible field authorizer after receiving the query result. According to the rules, the result desensitization processor carries out desensitization processing on the query result, prevents sensitive information from being leaked, and ensures privacy protection of data in the transmission and use processes.
It should be noted that the foregoing examples are only for understanding the present application, and do not constitute a limitation on the method for processing the API data service visible domain of the large data platform of the present application, and it is within the scope of the present application to perform more forms of simple transformation based on the technical concept.
The application provides a large data platform API data service visible domain processing device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions which can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large data platform API data service visible domain processing method in the first embodiment.
Referring now to FIG. 8, a diagram illustrating the architecture of a large data platform API data service visibility field processing device suitable for use in implementing an embodiment of the present application is shown. The large data platform API data service visible domain processing device in the embodiment of the present application may include, but is not limited to, mobile terminals such as notebook computers, PAD (Portable Application Description, tablet computers) and the like, and fixed terminals such as desktop computers and the like. The large data platform API data service visible domain processing device shown in fig. 8 is merely an example, and should not impose any limitation on the functionality and scope of use of embodiments of the present application.
As shown in fig. 8, the large data platform API data service visible domain processing apparatus may include a processing device 1001 (e.g., a central processor, a graphic processor, etc.), which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access Memory (RAM, random Access Memory) 1004. In the random access memory 1004, various programs and data required for the operation of the large data platform API data service visible domain processing apparatus are also stored. The processing device 1001, the read only memory 1002, and the random access memory 1004 are connected to each other by a bus 1005. An input/output (I/O) interface 1006 is also connected to the bus. In general, a system including an input device 1007 such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc., an output device 1008 including a Liquid Crystal Display (LCD) CRYSTAL DISPLAY, a speaker, a vibrator, etc., a storage device 1003 including a magnetic tape, a hard disk, etc., and a communication device 1009 may be connected to the I/O interface 1006. The communicator 1009 may allow the big data platform API data service visible domain processing device to communicate wirelessly or by wire with other devices to exchange data. While a large data platform API data service visible domain processing device is shown with various systems, it should be understood that not all of the illustrated systems are required to be implemented or provided. More or fewer systems may alternatively be implemented or provided.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via a communication device, or installed from the storage device 1003, or installed from the read only memory 1002. The above-described functions defined in the method of the disclosed embodiment of the application are performed when the computer program is executed by the processing device 1001.
The large data platform API data service visible domain processing equipment provided by the application adopts the large data platform API data service visible domain processing method in the embodiment, and can solve the technical problem of high maintenance cost of the API data service in the current data access control. Compared with the prior art, the beneficial effects of the large data platform API data service visible domain processing device provided by the application are the same as those of the large data platform API data service visible domain processing method provided by the embodiment, and other technical features in the large data platform API data service visible domain processing device are the same as those disclosed in the method of the previous embodiment, and are not repeated herein.
It is to be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon for performing the large data platform API data service visibility field processing method of the above-described embodiments.
The computer readable storage medium provided by the present application may be, for example, a U disk, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access Memory, a Read-Only Memory, an erasable programmable Read-Only Memory (EPROM, erasable Programmable Read Only Memory) or flash Memory, an optical fiber, a portable compact disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this embodiment, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to electrical wiring, fiber optic cable, radio Frequency (RF), the like, or any suitable combination of the foregoing.
The computer readable storage medium may be included in the large data platform API data service visible domain processing apparatus or may exist alone without being assembled into the large data platform API data service visible domain processing apparatus.
The computer readable storage medium carries one or more programs which, when executed by the large data platform API data service visibility field processing device, cause the large data platform API data service visibility field processing device to write computer program code for performing operations of the present application in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, or as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN, local Area Network) or a wide area network (WAN, wide Area Network), or may be connected to an external computer (e.g., through the internet using an internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present application may be implemented in software or in hardware. Wherein the name of the module does not constitute a limitation of the unit itself in some cases.
The readable storage medium provided by the application is a computer readable storage medium, and the computer readable storage medium stores computer readable program instructions (namely computer program) for executing the method for processing the API data service visible domain of the big data platform, so that the technical problem of high maintenance cost of the API data service in the current data access control can be solved. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the application are the same as the beneficial effects of the large data platform API data service visible domain processing method provided by the embodiment, and are not repeated here.
The foregoing description is only a partial embodiment of the present application, and is not intended to limit the scope of the present application, and all the equivalent structural changes made by the description and the accompanying drawings under the technical concept of the present application, or the direct/indirect application in other related technical fields are included in the scope of the present application.
Claims (7)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510457594.7A CN120017712B (en) | 2025-04-14 | 2025-04-14 | Method, equipment and storage medium for processing large data platform API data service visible domain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510457594.7A CN120017712B (en) | 2025-04-14 | 2025-04-14 | Method, equipment and storage medium for processing large data platform API data service visible domain |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN120017712A CN120017712A (en) | 2025-05-16 |
| CN120017712B true CN120017712B (en) | 2025-07-08 |
Family
ID=95664589
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510457594.7A Active CN120017712B (en) | 2025-04-14 | 2025-04-14 | Method, equipment and storage medium for processing large data platform API data service visible domain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120017712B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626870A (en) * | 2021-08-19 | 2021-11-09 | 微民保险代理有限公司 | Access control method, device, electronic equipment and storage medium |
| CN116975893A (en) * | 2023-08-14 | 2023-10-31 | 康键信息技术（深圳）有限公司 | Access request processing method and device, storage medium and computer equipment |
| CN119149788A (en) * | 2024-07-30 | 2024-12-17 | 马上消费金融股份有限公司 | Resource access control method, device, electronic equipment, storage medium and program product |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9135585B2 (en) * | 2010-06-15 | 2015-09-15 | Sap Se | Managing consistent interfaces for property library, property list template, quantity conversion virtual object, and supplier property specification business objects across heterogeneous systems |
| US10185726B2 (en) * | 2016-08-26 | 2019-01-22 | BlueTalon, Inc. | Access control for nested data fields |
| CN115729912A (en) * | 2022-11-08 | 2023-03-03 | 杭州趣链科技有限公司 | Database access system, database access method, computer device, and storage medium |
-
2025
- 2025-04-14 CN CN202510457594.7A patent/CN120017712B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626870A (en) * | 2021-08-19 | 2021-11-09 | 微民保险代理有限公司 | Access control method, device, electronic equipment and storage medium |
| CN116975893A (en) * | 2023-08-14 | 2023-10-31 | 康键信息技术（深圳）有限公司 | Access request processing method and device, storage medium and computer equipment |
| CN119149788A (en) * | 2024-07-30 | 2024-12-17 | 马上消费金融股份有限公司 | Resource access control method, device, electronic equipment, storage medium and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN120017712A (en) | 2025-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11855995B2 (en) | Data security across data residency restriction boundaries | |
| US11106820B2 (en) | Data anonymization | |
| AU2017351024B2 (en) | Processing application programming interface (API) queries based on variable schemas | |
| US11362997B2 (en) | Real-time policy rule evaluation with multistage processing | |
| US10956232B2 (en) | Instructing the use of application programming interface commands in a runtime environment | |
| US10834059B2 (en) | Secure message handling of an application across deployment locations | |
| US20230259406A1 (en) | Workflow Data Redistribution in Hybrid Public/Private Computing Environments | |
| US12130721B2 (en) | Data governance policy recommendations for application program interfaces | |
| CN111125064A (en) | Method and device for generating database mode definition statement | |
| CN117009397A (en) | Data query method, data query device, electronic equipment and storage medium | |
| US11288170B1 (en) | Log analysis debugging without running on real production environment | |
| US10762088B2 (en) | Automated database manager | |
| US11954231B2 (en) | Recursively adapting a sensitive content masking technique | |
| CN120017712B (en) | Method, equipment and storage medium for processing large data platform API data service visible domain | |
| US12301582B2 (en) | Tenant configuration state extraction and variance detection | |
| US12039069B2 (en) | Metadata-driven restricted measures | |
| US12141313B2 (en) | Security semantics for database queries | |
| US10620946B1 (en) | Dynamic modeling for opaque code during static analysis | |
| CN115827589A (en) | A permission verification method, device, electronic equipment and storage medium | |
| WO2024010662A1 (en) | Tenant configuration state extraction and variance detection | |
| US20220188103A1 (en) | Subject matter expert identification and code analysis based on a probabilistic filter | |
| CN117349891A (en) | Dynamic data authority control method, device and medium based on Mybatis | |
| CN117827922A (en) | Data service method and device based on data asset and electronic equipment | |
| CN114780540A (en) | Metadata processing method and device for big data platform and electronic equipment | |
| CN120671156A (en) | A permission verification method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |