CN119493665A

CN119493665A - Data processing method, device, equipment and medium based on layered architecture

Info

Publication number: CN119493665A
Application number: CN202411550876.3A
Authority: CN
Inventors: 吴明宝; 黄潇豪
Original assignee: Ping An Life Insurance Company of China Ltd
Current assignee: Ping An Life Insurance Company of China Ltd
Priority date: 2024-11-01
Filing date: 2024-11-01
Publication date: 2025-02-21

Abstract

The present invention relates to the fields of data processing technology and financial technology, and discloses a data processing method based on a layered architecture, by dividing a data layer and a control layer, deploying a concurrent data processing module in the data layer, and configuring concurrent connection parameters with a distributed cache module; deploying a request control module in the control layer, and configuring control connection parameters with the distributed cache module; the control layer receives a request from an application component, determines that the request type is a common request or a high-concurrency request, and when processing a common request, accesses the target information in the distributed cache module through an access interface of the control layer, and when processing a high-concurrency request, sends the request to the data layer, and accesses the target information in the distributed cache module through the concurrent data processing module. The present invention divides the data processing architecture into a data layer and a control layer, so that high-concurrency requests can be directly processed through the data layer, which greatly reduces the number of application components and reduces resource waste.

Description

Data processing method, device, equipment and medium based on layered architecture

Technical Field

The present invention relates to the field of data processing technologies and financial technologies, and in particular, to a data processing method, device, equipment and storage medium based on a hierarchical architecture.

Background

In the financial field, with the acceleration of the digitizing process, the amount of concurrent data requests faced by financial institutions is increasing, and particularly, during large-scale sales promotion, financial settlement peak or market fluctuation, a large number of user requests are intensively gushed in, resulting in high concurrent pressure. Conventional data processing architectures typically handle concurrent requests of users in common through business application components and middleware components. However, this mode has the following disadvantages:

Business application components are mostly implemented using programming languages such as Java, go, python, which, while easy to program, have relatively low performance and include garbage collection mechanisms (GC), which perform poorly in high concurrency scenarios. In contrast, middleware components are typically developed using high performance, GC-free programming languages such as C, C ++ or Rust, which typically have processing power more than 5 times that of business application components. However, under conventional architecture, each request must go through business application components and middleware, which results in a dramatic increase in the number and load of business application components.

In financial transactions, campaign promotions and market fluctuations are frequent, and financial institutions often need temporary extension of the transaction application components to handle high concurrency requests. Although the pressure in the peak period can be relieved through dynamic capacity expansion, the strategy needs to prepare a large number of physical machines in advance, so that the resource pool is high in demand and high in maintenance cost, and a large number of resources are in an idle state at ordinary times, so that great resource waste is caused.

Because the traffic of each request needs to pass through the service application component and the middleware in turn, unnecessary traffic transfer and additional processing on the network level are caused, and the architecture increases the delay of information access and influences the access experience of users and the response efficiency of the system.

Traditional architectures have to share the load by adding a large number of business application components in a high concurrency environment, which results in a rapid expansion of the number of physical machines, system management and maintenance becomes more complex, and is detrimental to cost-effective control and overall stability of the system.

Disclosure of Invention

The invention mainly aims to provide a data processing method, device, equipment and storage medium based on a layered architecture, and aims to solve the technical problems of serious resource waste and high system expansion cost caused by the expansion of the number of service application components in a high concurrency scene in the prior art.

In order to achieve the above object, the present invention provides a data processing method based on a hierarchical architecture, including:

dividing a data processing architecture into a data layer and a control layer;

Deploying a concurrent data processing module in the data layer, and configuring concurrent connection parameters of the concurrent data processing module and a distributed cache module;

in the control layer, a request control module is deployed, and control connection parameters of the request control module and the distributed cache module are configured;

In the control layer, receiving a request from an application component, and determining a type of the request, wherein the type of the request comprises a common request or a high concurrency request;

If the request is a common request, accessing target information from the distributed cache module according to the control connection parameters through an access interface provided by a request control module of the control layer, and returning the target information to an application component which initiates the request;

And if the request is a high concurrency request, sending the high concurrency request to the data layer, accessing target information from the distributed cache module through a concurrency data processing module of the data layer according to the concurrency connection parameters, and returning the target information to an application component which initiates the request.

Further, to achieve the above object, the present invention provides a data processing apparatus based on a hierarchical architecture, including:

The architecture management module is used for dividing the data processing architecture into a data layer and a control layer;

The concurrent data processing module is used for deploying the concurrent data processing module in the data layer and configuring concurrent connection parameters of the concurrent data processing module and the distributed cache module;

The request control module is used for deploying the request control module in the control layer and configuring control connection parameters of the request control module and the distributed cache module;

A request identification module, configured to receive a request from an application component in the control layer, and determine a type of the request, where the type of the request includes a normal request or a high concurrency request;

The common request processing module is used for accessing target information from the distributed cache module according to the control connection parameters through an access interface provided by a request control module of the control layer if the request is a common request, and returning the target information to an application component which initiates the request;

And the concurrency data processing module is used for sending the high concurrency request to the data layer if the request is a high concurrency request, accessing target information from the distributed cache module according to the concurrency connection parameter through the concurrency data processing module of the data layer, and returning the target information to an application component which initiates the request.

Further, to achieve the above object, the present invention also provides a computer device, which includes a memory, a processor, and a hierarchical architecture-based data processing program stored on the memory and executable on the processor, where the hierarchical architecture-based data processing program implements the steps of the hierarchical architecture-based data processing method as described above when executed by the processor.

Further, to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a hierarchical architecture-based data processing program which, when executed by a processor, implements the steps of the hierarchical architecture-based data processing method as described above.

The invention has the beneficial effects that the invention relates to the technical field of data processing and the technical field of finance and discloses a data processing method based on a layered architecture, wherein concurrent data processing modules are arranged in the data layer by dividing the data layer and a control layer, concurrent connection parameters with a distributed cache module are configured, a request control module is arranged in the control layer, control connection parameters with the distributed cache module are configured, the control layer receives a request from an application component, the request type is determined to be a common request or a high concurrent request, when the common request is processed, target information in the distributed cache module is accessed through an access interface of the control layer, when the high concurrent request is processed, the request is sent to the data layer, and the target information in the distributed cache module is accessed through the concurrent data processing module. According to the invention, the data processing architecture is divided into the data layer and the control layer, so that the high concurrency request can be directly processed through the data layer, the number of application components is greatly reduced, and the resource waste is reduced. Meanwhile, through modularized management of different layers, the data access path is optimized, delay is reduced, and the overall performance and expansibility of the system are improved.

Drawings

The invention will be further described with reference to the accompanying drawings and examples, in which:

FIG. 1 is a schematic diagram of an application environment of a data processing method based on a hierarchical architecture according to an embodiment of the present invention;

FIG. 2 is a flow chart of a data processing method based on a hierarchical architecture according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of functional blocks of a data processing apparatus according to a preferred embodiment of the present invention;

FIG. 4 is a schematic diagram of a computer device according to an embodiment of the invention;

fig. 5 is a schematic diagram of another structure of a computer device according to an embodiment of the invention.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

The data processing method based on the layered architecture provided by the embodiment of the invention can be applied to an application environment as shown in fig. 1, wherein a user side communicates with a server side through a network. The server side can divide the data processing architecture into a data layer and a control layer through the user side, a concurrent data processing module is arranged in the data layer, concurrent connection parameters of the distributed cache module are configured, a request control module is arranged in the control layer, control connection parameters of the distributed cache module are configured, the control layer receives a request from an application component, the type of the request is determined to be a common request or a high concurrent request, when the common request is processed, target information in the distributed cache module is accessed through an access interface of the control layer, when the high concurrent request is processed, the request is sent to the data layer, and target information in the distributed cache module is accessed through the concurrent data processing module. According to the invention, the data processing architecture is divided into the data layer and the control layer, so that the high concurrency request can be directly processed through the data layer, the number of application components is greatly reduced, and the resource waste is reduced. Meanwhile, through modularized management of different layers, the data access path is optimized, delay is reduced, and the overall performance and expansibility of the system are improved. The user terminal may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server may be implemented by a stand-alone server or a server cluster formed by a plurality of servers. The present invention will be described in detail with reference to specific examples.

Referring to fig. 2, fig. 2 is a flowchart illustrating an embodiment of a data processing method based on a hierarchical architecture according to the present invention. It should be noted that although a logical order is depicted in the flowchart, in some cases the steps depicted or described may be performed in a different order than presented herein.

As shown in fig. 2, the data processing method based on the layered architecture provided by the invention comprises the following steps:

s10, dividing a data processing architecture into a data layer and a control layer;

In this embodiment, the data processing architecture is designed to optimize the data processing efficiency and simplify the management of the system, and the responsibility of data processing is clearly divided into two major parts, namely a data layer and a control layer. The data layer is mainly responsible for data access, data caching, data calculation and large-scale concurrent processing in a high concurrency environment. The control layer is mainly responsible for receiving, classifying and forwarding the request and integrating with the service logic of the upper layer.

The data layer is focused on handling high frequency, high concurrency data access requests, and data processing capacity is improved by using high performance programming languages and optimized concurrency control means. The data layer is designed as a separate module to reduce the burden of application components to directly handle high concurrency requests.

The control layer is responsible for receiving requests from application components and performing classification and scheduling management of the requests. The design of the control layer can effectively reduce the complexity of direct data operation, and the request circulation is optimized through a control and classification mechanism.

The data layer and the control layer are mutually independent and arranged in a framework, and interact and synchronize data through high-speed network connection or shared distributed cache modules. Thus, the independence of the two layers is ensured, and the two layers can be independently expanded or optimized according to the actual load condition.

In the design stage of the system architecture, responsibilities of a data layer and a control layer are definitely divided according to the access frequency of data, the complexity of business logic and the concurrent load condition. The high concurrency processing tasks are concentrated to the data layer, and the receipt, classification, and scheduling of requests are concentrated to the control layer. In the system deployment stage, the concurrent processing module of the data layer is deployed on a high-performance physical server or virtual machine according to a preset partitioning strategy, and the request control module of the control layer is deployed on another group of servers, so that the two layers are ensured to be physically separated. An explicit communication protocol and data interaction interface is defined between the data layer and the control layer to ensure efficient collaboration and data consistency between the two layers. This may be achieved by high-speed network connection, cache sharing or RPC (remote procedure call) techniques, etc.

It is illustrated that in a large scale transaction processing scenario in the financial field, such as in an online payment system of a bank, it is often necessary to process concurrent transaction requests of a large number of users. By dividing the data processing architecture into a data layer and a control layer, the control layer is responsible for receiving and parsing transaction requests from clients, the data layer specifically handles the storage and querying of transaction records. In this way, the control layer can efficiently sort and schedule requests, and pass highly concurrent requests to the data layer for processing, thereby reducing system delay and improving throughput of data processing.

By dividing the data processing architecture into a data layer and a control layer, the module design of the system can be effectively simplified, and the data processing efficiency and the system expansibility can be improved. The data layer is focused on data access in a high concurrency scene, the burden of directly processing the high concurrency by an application component is reduced, and the control layer is responsible for receiving and scheduling requests, so that system delay caused by complex data processing is reduced. The hierarchical architecture can reduce the number of components, optimize the resource utilization, and make the expansion and maintenance of the system more flexible.

S20, deploying a concurrent data processing module in the data layer, and configuring concurrent connection parameters of the concurrent data processing module and a distributed cache module;

in this embodiment, the concurrent data processing module is deployed in the data layer, mainly for processing highly concurrent data requests. The core of the module design is to improve the efficiency of data access and avoid system bottleneck and performance degradation caused by high concurrent access. Concurrent data processing modules are typically written using high performance languages (e.g., c++, rust) to optimize concurrent data access capabilities.

The concurrent connection parameters include a plurality of configuration items such as a size of a connection pool, a timeout time of each connection, a maximum number of concurrent connections, and a number of retries, etc. The configuration of the parameters aims at optimizing the concurrent connection between the module and the distributed cache module and improving the reading and writing efficiency and response speed of data.

The concurrent data processing module needs to establish multiple concurrent connections with the distributed cache module in order to handle a large number of concurrent data access requests in a high concurrency environment. The purpose of configuring the connections is to control the size of the connection pool, the lifetime of each connection (configured by timeout time), and retry policies when the connection fails, etc.

First, a concurrent data processing module is installed on a server or virtual machine of a data layer. The installation of the module may be accomplished by an automated deployment tool, such as using a containerization technique (e.g., docker) to deploy the module to different nodes to ensure extensibility of the module. After the installation is completed, setting connection parameters for the concurrent data processing module through a configuration file or a management platform. Typical configuration items include:

and the connection pool size is used for defining the maximum concurrent connection number between the data layer and the distributed cache module.

And the overtime time is used for setting the maximum survival time of each connection and preventing the connection resources from being occupied for a long time.

The retry times are used for setting the retry times and strategies after connection failure or overtime.

When the concurrent data processing module is initialized, a plurality of connections are automatically established with the distributed cache module according to preset concurrent connection parameters, and the connection resources are uniformly managed. The module can deal with the continuous variable concurrent load by dynamically distributing and recycling the connection resources, so that the data processing efficiency in the high concurrent environment is ensured.

In financial data processing, such as a stock exchange system, a concurrent data processing module may quickly process concurrent order requests in a data layer when a large number of users submit sales orders simultaneously. By configuring 1000 concurrent connections and setting reasonable timeout time and retry strategy, the module can ensure that each order request can be received and processed quickly, thereby reducing waiting time of users and improving response speed and processing capacity of the system.

By arranging the concurrent data processing module in the data layer and reasonably configuring the concurrent connection parameters, the data processing efficiency in the high concurrent environment can be effectively improved. The reasonable management and dynamic adjustment mechanism of the connection pool ensures the stability and high efficiency of the system when processing a large number of concurrent requests. And the optimized connection strategy with the distributed cache module can reduce the delay of data access and improve the throughput of overall data processing.

S30, deploying a request control module in the control layer, and configuring control connection parameters of the request control module and the distributed cache module;

In this embodiment, the control layer is mainly responsible for the reception, classification and scheduling of requests. The request control module is a core module deployed in the control layer for listening for requests from various application components. The module analyzes and classifies the requests through preset rules or algorithms and delivers different types of requests to a proper processing module for processing.

The request control module needs to establish a connection with the distributed cache module so as to be capable of directly acquiring data from the distributed cache when processing a common request. The control connection parameters include the size of the connection pool, the maximum number of concurrency per connection, the timeout time, etc. The configuration of these parameters is mainly aimed at the processing requirements of common requests, focusing on the stability of the connection and the utilization of resources.

The request control module manages the connection with the distributed cache module according to the set control connection parameters. The method aims to provide efficient and stable connection management under the condition of common requests, and avoid excessive pressure on the access of the distributed cache module.

And installing a request control module on a physical server or a virtual machine of the control layer, wherein the request control module is mainly deployed through a standard service installation mode. Typically, the installation of the modules may be accomplished by automated tools (e.g., ansible, kubernetes, etc.) to simplify the deployment process. After deployment is completed, setting control connection parameters for the request control module through a configuration file or a system management platform. These parameters include:

And the connection pool size is used for designating the maximum connection number between the control layer and the distributed cache module and is used for coping with the concurrent access of the common request.

And limiting the number of common requests processed simultaneously in the control layer to prevent overload of the requests from affecting the stability of the control layer.

And setting the maximum survival time of each connection in the control layer to ensure the stability and timely recovery of the connection.

And when the request control module is initialized, a plurality of stable connections are established with the distributed cache module according to preset control connection parameters. The modules dynamically allocate and manage these connection resources according to the access requirements of the normal requests, thereby achieving efficient data access in the processing of the requests.

In banking systems in the financial field, a number of general query requests, such as account balance queries or transaction record queries, are typically processed. The common requests can be responded quickly by deploying the request control module at the control layer and configuring the control connection parameters with the distributed cache module. The reasonable setting of the connection pools (such as 10 connection pools) is controlled, and the proper timeout time and the maximum concurrency number configuration are combined, so that the system resources can be ensured to be efficiently utilized in the process of processing the common request, and the problem of response delay caused by unstable connection is avoided.

The processing efficiency and stability of the common request can be remarkably improved by arranging the request control module in the control layer and reasonably configuring the control connection parameters. The request control module is focused on receiving and managing the common requests, and can flexibly select proper connection resources according to the request types, so that the influence of a control layer on high-concurrency request processing is reduced. In addition, through parameterized configuration, flexible connection management can be performed aiming at different service scenes, and the expansibility and reliability of a control layer are improved.

S40, in the control layer, receiving a request from an application component, and determining the type of the request, wherein the type of the request comprises a common request or a high concurrency request;

In this embodiment, one of the core responsibilities of the control layer is to receive various types of requests from application components. An application component is the interface of a client with the system that translates user operations (e.g., querying, submitting, updating, etc.) into system requests and sends them to the control layer. The request control module of the control layer receives the requests through a monitoring network port or an API interface, and extracts key information through preliminary analysis.

After receiving the request, the request control module analyzes the content and parameters of the request. The parsed content includes the type of request (e.g., read, write), the priority of the request, the source information of the request, etc. The purpose of this step is to obtain enough information to prepare for subsequent request classification.

The classification of the request is mainly based on preset rules or policies. The control layer may divide the request into "normal requests" or "high concurrency requests" depending on the content and nature of the request. Common requests generally refer to requests requiring less computational resources and lower priority, while highly concurrent requests involve large amounts of data processing and high priority requests. The classification of requests is typically accomplished using a rules engine or simple logic judgment.

The classification of a request is based on factors such as the type of request (e.g., query, write, etc.), the size of the data in the request body, the priority of the request (e.g., VIP user request), etc. By these sorting bases, the system is able to dynamically determine the request type in order to select the appropriate processing path.

For example, the classification rule of the request may be set based on the data size of the request:

Common requests-the amount of data for a single request is small, typically in the hundreds to thousands of bytes. For example, the amount of data requested to query account balances, obtain user base information, etc. is small.

High concurrency requests-the amount of data for a single request is large, possibly reaching hundreds of KB or even several MB. Such as querying a large number of transaction records, obtaining detailed statistical analysis data, etc.

Requests that have a single request data size exceeding 50KB can be considered high concurrency requests.

The classification rule of the request may be set based on the processing time of the request:

Common requests, with expected processing times being short, typically between tens of milliseconds and hundreds of milliseconds. Such as a request to query the user for basic information or account balance.

Highly concurrent requests, which take a long time to complete, typically exceed 500 milliseconds. For example, to process large volumes of order data or to perform complex computing and data aggregation operations.

Requests that have a processing time of a single request exceeding 500 milliseconds may be considered high concurrency requests.

The classification rules for the requests may also be set based on the concurrency number of requests:

the common request has relatively low request concurrency, and can be stabilized within hundreds. Such as user sporadic query operations or off-peak basic data requests.

High concurrency request, high concurrency number, up to thousands or tens of thousands. Such as a large promotional campaign or peak request during market surge.

Requests with a concurrency number exceeding 500 may be considered high concurrency requests.

The classification rules for the requests may also be set based on the priority and source of the requests:

General requests, requests having a lower priority, are typically from general users or regular automated tasks. For example, daily account checking tasks.

High concurrency requests, where the priority of the request is high, may come from batch tasks, mass promotional campaigns, or batch requests by VIP users.

For bulk requests from a particular group of users (e.g., VIP users) or a particular event (e.g., promotional event), high concurrency requests may be considered.

In practical applications, the system may comprehensively determine whether a request is a high concurrency request according to a combination of multiple quantization criteria. The following is a partial rule example:

rule 1 if the amount of data for a single request exceeds 50KB, then high concurrency requests are deemed.

Rule 2 if the processing time of a single request exceeds 500 milliseconds, then a high concurrency request is deemed.

Rule 3 if the number of concurrent requests at the same time exceeds 500, then high concurrent requests are considered.

Rule 4 if the source of the request is a batch task or a large batch request by a VIP user, then a high concurrency request is considered.

The request control module monitors a network interface or an open API endpoint at startup for receiving requests from application components in real time. When a request arrives, the control module first confirms the validity of its source and places it in the request queue. After the request enters the control module, the system extracts the core parameters of the request by analyzing the information in the request header and the request body. These parameters include operation type, amount of requested data, priority flags, etc. After the analysis is completed, the information is passed as input to classification logic. The request control module classifies the request into a common request or a high concurrency request according to the analyzed parameters through a preset classification rule (such as judging the size of the request data quantity, the complexity of the operation type and the like). The result of the classification will determine the processing path of the subsequent request.

In a customer transaction inquiry system in the financial field, the inquiry request of a user can be divided into a common account balance inquiry and a transaction detail inquiry which needs to process a large amount of data. Through a request control module in the control layer, the system can classify received requests, treat account balance queries as normal requests, and treat transaction detail queries as high concurrency requests. Through the classification, the common requests can be directly and quickly responded in the control layer, and the high concurrency requests are forwarded to the data layer for parallel processing, so that the response speed and the processing efficiency of the system are improved.

By receiving and analyzing the requests in the control layer and classifying the requests, the processing efficiency of the system on different types of requests can be remarkably improved. Through dynamic classification, the priority and the processing path of the request can be flexibly adjusted according to the current load state of the system, so that reasonable utilization of system resources is ensured, and system congestion is avoided under the condition of high concurrency. At the same time, the classification processing mode also enables the high concurrency request to be exclusively processed by the data layer, and the common request can be directly completed in the control layer.

S50, if the request is a common request, accessing target information from the distributed cache module according to the control connection parameters through an access interface provided by a request control module of the control layer, and returning the target information to an application component which initiates the request;

In this embodiment, after receiving the request, the request control module in the control layer determines that the type of the current request is a common request according to a preset rule or a classification algorithm. Common requests typically include requests with a smaller amount of data and a lower computational burden, such as account balance queries, basic information queries, and the like.

And the request control module calls the configured standardized access interface to process the common request according to the judging result. These access interfaces typically interface directly with the distributed cache modules to quickly obtain the target data. The design of interface calls follows a simple and efficient principle to reduce unnecessary processing steps and delays.

The control connection parameters are used to control connection management policies of the access interface, such as the size of the connection pool, the timeout time of a single connection, the maximum number of retries, etc. The request control module, when invoking the access interface, will refer to these parameters to dynamically allocate and manage the connections with the distributed cache modules.

And establishing connection between the request control module and the distributed cache module through an access interface, and inquiring target information from the cache according to the content of the common request. The distributed cache module generally stores data accessed at high frequency to improve the access efficiency and response speed of the data.

After the target information is successfully acquired, the control module sends the target information back to the application component which initiates the request according to the return path and the format in the request, and the whole processing flow of the common request is completed.

In the request control module, the system judges whether the current request is a common request according to a preset rule. If it is a normal request, the system will select the appropriate access interface to process. The control module calls a preconfigured access interface according to the type of the common request. Before the access interface is called, an idle connection is distributed from the connection pool according to the control connection parameters, and timeout time and retry strategy are set. The access interface interacts with the distributed cache module through the distributed connection to inquire the target information. If there is target information in the cache, return is immediate. If the target information is not present in the cache, further processing may be required depending on the configuration. After successfully obtaining the target information, the access interface sends the data to the requesting application component via a predetermined return path. After the data return is completed, the control module releases the allocated connection and returns it to the connection pool to service the next request.

In a credit card bill inquiry system in the financial field, a user generally inquires about basic information such as current account balance, credit line and the like. The data volume of the query operations is small, the processing time is short, and the query operations belong to common requests. Through the request control module of the control layer, the system can quickly identify and classify the requests, and through the standardized access interface, related information is queried from the distributed cache module, and data required by a user is quickly returned, so that user experience is remarkably improved and load pressure of the system is reduced.

By providing a standardized access interface for the common request, the processing overhead of the system on the common request can be obviously reduced, and the response speed is improved. Through reasonable connection parameter configuration, the control module can flexibly manage connection resources of the distributed cache module, and waste of the connection resources in a common request scene is avoided. Meanwhile, the stability and the expandability of the system can be improved through layering processing and standardized design of an access interface.

And S60, if the request is a high concurrency request, sending the high concurrency request to the data layer, accessing target information from the distributed cache module through a concurrency data processing module of the data layer according to the concurrency connection parameters, and returning the target information to an application component which initiates the request.

In this embodiment, in the control layer, the request control module may determine and identify whether the current request is a high concurrency request based on the characteristics of the request, the data volume, the user priority, or the current load state of the system. High concurrency requests typically involve operations with large amounts of data, high concurrency numbers, and long processing times.

After determining that the request is a high concurrency request, the request control module sends the request to a concurrency data processing module of the data layer through a designated network path or a dedicated interface. The data layer is focused on handling a large number of concurrent requests and is therefore designed to support high throughput and parallel data processing.

And after receiving the high concurrency request, the concurrency data processing module of the data layer processes according to preset concurrency connection parameters. These parameters include the size of the connection pool, timeout time, maximum concurrency, etc. The concurrency data processing module allocates appropriate connections from the initialized connection pool to interact with the distributed cache module according to the parameters.

And the concurrent data processing module establishes a parallel connection channel with the distributed cache module through the distributed connection. The module may obtain the target information from the cache using an efficient data access algorithm based on the specific content of the request. If there is no target information in the cache, a subsequent data acquisition policy may also be triggered.

After the target information is successfully acquired, the data layer sends the target information back to the application component which initiates the request according to the return path and the data format in the request. Meanwhile, the concurrency data processing module releases the used connection resources and returns the connection resources to the connection pool so as to ensure the recycling of the resources.

The request control module judges that the current request is a high concurrency request through a rule engine or a preset classification rule, and then forwards the request to a concurrency data processing module of the data layer by utilizing a special high concurrency interface. When forwarding, the module will attach the critical parameters of the request and return path information. After receiving the high concurrency request, the concurrency data processing module allocates proper connection resources from the connection pool according to the concurrency connection parameters. The module can decide how many concurrent connections to use according to the content and data volume of the request to optimize the processing efficiency of the concurrent request. And the concurrent data processing module reads the target information from the distributed cache module by using an efficient data access strategy through the distributed connection. In a high concurrency scenario, the module may enable optimization techniques such as parallel reading and batch reading to ensure stability and efficiency of data access. After the data access is completed, the module returns the target information to the requesting application component along the path in the request. The module then releases and recovers the used connection resources, ensuring the stability and availability of the connection pool.

In an online payment system in the financial domain, a large number of highly concurrent requests occur each time a large-scale promotional program is conducted or a large number of transactions are processed. The control layer can quickly identify these high concurrency requests and forward them to the data layer for processing. The concurrent data processing module of the data layer rapidly allocates connection resources according to the configured concurrent connection parameters, and establishes an efficient data channel with the distributed cache module, so that a large number of concurrent transaction requests are processed in a short time, the waiting time of a user is reduced, and the overall response speed of the system is improved.

The high concurrency request is forwarded to the data layer for processing, so that the load of the control layer can be effectively dispersed, and the expansibility of the system is improved. Meanwhile, the concurrent data processing module can flexibly manage connection resources between the concurrent data processing module and the distributed cache module according to preset concurrent connection parameters, and stability and high efficiency of data access in a high concurrent environment are ensured. Through hierarchical architecture design and optimized scheduling of resources, the invention can obviously reduce response delay of a system and improve the overall throughput of concurrent data processing.

The invention relates to the technical field of data processing and the field of financial science and technology, and discloses a data processing method based on a layered architecture, wherein a concurrent data processing module is arranged in a data layer by dividing the data layer and a control layer, and concurrent connection parameters of the concurrent data processing module and a distributed cache module are configured; the control layer receives the request from the application component, determines the request type as a common request or a high concurrency request, accesses the target information in the distributed cache module through an access interface of the control layer when the common request is processed, and sends the request to the data layer when the high concurrency request is processed, and accesses the target information in the distributed cache module through the concurrency data processing module. According to the invention, the data processing architecture is divided into the data layer and the control layer, so that the high concurrency request can be directly processed through the data layer, the number of application components is greatly reduced, and the resource waste is reduced. Meanwhile, through modularized management of different layers, the data access path is optimized, delay is reduced, and the overall performance and expansibility of the system are improved.

In one embodiment, in S60, after accessing, by the concurrent data processing module of the data layer, the target information from the distributed cache module according to the concurrent connection parameter, the method further includes:

S701, when the concurrent data processing module fails to access target information from the distributed cache module, acquiring the target information from a database through the concurrent data processing module;

S702, caching the target information into the distributed caching module;

And S703, returning the target information to the application component which initiates the request.

In this embodiment, the concurrent data processing module of the data layer is utilized to directly obtain the target information from the distributed cache module according to the concurrent connection parameters (including the size of the connection pool, the timeout time, the retry number, etc.). The concurrent data processing module distributes connection from a pre-configured connection pool, establishes a high-speed connection channel with the distributed cache module, and rapidly inquires target information through a parallel access technology.

In the process of the concurrent data processing module attempting to acquire the target information from the distributed cache module, access failure may be caused by cache miss, data failure or connection failure. At this point, the concurrent data processing module needs to have the ability to detect these anomalies and take remedial action.

When the access to the distributed cache module fails, the concurrent data processing module switches to a standby database data source and re-queries the target information from the database. The data sources in the database are considered authoritative and persistent, so that when a cache misses, the module will directly retrieve the latest target information from the database.

After the target information is successfully obtained from the database, the concurrent data processing module writes the information into the distributed cache module so that the information can be directly obtained from the cache when the information is accessed next time, and the processing efficiency of subsequent requests is improved. The step can avoid the data from being queried from the database for multiple times, and reduce the load pressure of the database.

And finally, returning the successfully acquired target information to the application component which initiates the request by the concurrency data processing module according to a preset path. And when the method returns, the delay of data transmission can be reduced through an efficient transmission protocol or compression technology, and the user experience is improved.

For example, the concurrent data processing module queries the distributed cache module for target information over the assigned connection attempt. In the access process, the module sets timeout time and controls retry times according to the connection parameters. If the data is not successfully acquired within a limited time, the access failure is determined. When the cache access fails, the module immediately switches to the database connection, and the query operation is re-executed from the database according to the target information of the request. The manner in which databases are accessed typically uses SQL or other optimized query techniques to reduce query time. After the target information is successfully acquired, the module updates the acquired data into the distributed cache module, and sets the cache time or expiration strategy of the data so as to ensure the timeliness and effectiveness of the cached data. The target information is then sent to the initiator of the request over the configured return path.

Through the steps, the embodiment can ensure that even if the distributed cache fails to provide effective data in a high concurrency environment, the latest target information can be timely acquired through the database. Meanwhile, the successfully acquired target information is written back into the distributed cache module, so that the cache hit rate of subsequent requests can be improved, the query pressure of a database is reduced, and the overall data access efficiency and the response speed of the system are improved.

In one embodiment, the step S20 includes:

S201, installing the concurrent data processing module on a physical server or a virtual machine of the data layer;

S202, setting concurrent connection parameters with the distributed cache module in the concurrent data processing module, wherein the concurrent connection parameters comprise a connection pool size, timeout time and retry times;

S203, initializing a connection pool according to the size of the connection pool, and pre-distributing available connection with the distributed cache module in the connection pool.

In this embodiment, the concurrent data processing module is installed on a hardware resource (such as a physical server or virtual machine) of the data layer. The installation of the module involves selecting appropriate hardware or virtualized resources to ensure computing power and network bandwidth to handle high concurrency requests.

In the concurrent data processing module, connection parameters with the distributed cache module are required to be set according to actual concurrent access requirements. These parameters include:

and the connection pool size is the maximum concurrent connection number allowed between the module and the distributed cache module.

Timeout, representing the maximum survival time of each connection, for preventing the connection from occupying resources for too long.

The retry number is the maximum retry number indicating connection failure or timeout, and is used for guaranteeing connection stability.

The connection pool is used for managing and distributing connection resources between a plurality of the connection pool and the distributed cache module. The process of initializing a connection pool includes creating and managing these connection resources according to the set connection pool size.

After the initialization is completed, the connection pool can pre-establish connection with the distributed cache module according to the configured size, and manage the connection as available resources. The pre-allocation aims to quickly allocate idle connection when processing a data access request and improve the efficiency of concurrent processing.

And installing a service program of the concurrent data processing module on a physical server or a virtual machine of the data layer. The installation process may be implemented by an automated deployment tool (e.g., ansible, kubernetes, etc.) that ensures that service instances of the modules are run on each physical server or virtual machine. And setting concurrent connection parameters with the distributed cache module in a configuration file of the module or through a management platform. The size of the connection pool determines the maximum number of allowed connections, the timeout is set to a reasonable length of time (e.g., 30 seconds) to avoid long-time connection occupation, and the retry number is set to 1-3 times to cope with connection failure. And when the module is started, initializing according to the set size of the connection pool, and creating a plurality of connection objects with the distributed cache module. The operation of initializing a connection pool typically includes pre-establishing a plurality of free connections with a cache module to enable quick response when processing a request. After initialization is completed, the connection pool will manage these connection resources in memory. Each connection object is marked as "available" during the initialization phase to ensure that idle connections can be quickly acquired by subsequent request processing.

According to the embodiment, the concurrent data processing module is arranged in the data layer, and the concurrent connection parameters with the distributed cache module are reasonably set, so that the efficiency of data access and the stability of connection can be effectively improved. The pre-distributed idle connection pool ensures the quick response capability under the high concurrency environment, and meanwhile, the stability of connection can be improved through reasonable timeout time and retry strategy, and the interruption of the system and the resource waste are reduced.

In one embodiment, in S60, accessing, by the concurrent data processing module of the data layer, the target information from the distributed cache module according to the concurrent connection parameter includes:

S601, determining idle connection from a pre-initialized connection pool through the concurrent data processing module, and executing data access operation in the distributed cache module through the idle connection;

s602, monitoring the running state of the current connection through the concurrent data processing module, and judging whether the current connection has connection timeout according to the timeout time in the concurrent connection parameter;

S603, if the concurrent data processing module detects that the current connection has overtime connection, releasing the current connection, and reestablishing connection with the distributed cache module according to retry times in the concurrent connection parameters;

S604, when the target information is successfully accessed, releasing and returning the used current connection to the connection pool through the concurrent data processing module.

In this embodiment, the concurrency data processing module searches for and determines a free connection from the initialized connection pool when processing the high concurrency request. Each connection in the connection pool has been initialized at the start-up of the module and a connection status flag is maintained to indicate whether the connection is "free" or "in use.

After finding the idle connection, the concurrent data processing module establishes an access channel with the distributed cache module through the connection, and executes data access operation. Data access operations typically include reading or writing target information in a cache.

And the concurrent data processing module is required to continuously monitor the running state of the current connection in the data access process. The monitoring of the operating state mainly relates to the validity, stability, response time, etc. of the connection. The monitoring aims to be able to handle in time when an abnormal situation is detected.

And the module judges whether the current connection completes the access operation within the preset time according to the timeout time set in the concurrent connection parameters. If the set time threshold is exceeded, a connection timeout is considered to occur.

If the module detects a connection timeout, the current connection is released immediately to avoid occupying limited connection resources. And then, according to the retry times in the concurrent connection parameters, trying to reestablish the connection with the distributed cache module, and ensuring the continuity and stability of data access.

After the data access is successfully completed, the concurrent data processing module releases the currently used connection and marks it as "idle" so that the next request can continue to use the connection. The process of releasing typically involves clearing the state and resources of the connection, ensuring stable and efficient operation of the connection pool.

And the concurrency data processing module determines idle connection according to the connection state mark by searching the connection object in the connection pool. In implementation, one available connection resource can be quickly obtained by traversing the connection pool or using the connection queue. After the idle connection is determined, the module sends a data access request through the connection, and returns target information according to the response of the distributed cache module. During access, the module records the usage status of the connection for processing upon timeout or failure. The module periodically checks the status of each connection by a timer or monitoring thread. If a certain connection exceeds a preset timeout period, the module immediately releases the connection to prevent the connection from occupying resources. Meanwhile, according to the set retry number, the connection is reassigned and access is attempted again. After successful completion of the data access, the module will re-mark the state of the current connection as "free" and return it to the connection pool. The connection release process also comprises resetting the connection object and cleaning the resources, so as to ensure the stability of the connection pool.

According to the embodiment, the idle connection is selected from the pre-initialized connection pool, the connection state is monitored in real time in the access process, and the system can remarkably improve the efficiency and the stability of high-concurrency data access. Meanwhile, through reasonable overtime time and retry strategy, the problems of long-time occupation of connection resources and interruption of data access can be effectively avoided. In addition, the stability and the persistence of the connection pool can be ensured through the release and the recovery of the resources after the successful access.

In one embodiment, the step S40 includes:

s401, monitoring and receiving a request from an application component through a request control module in the control layer;

s402, analyzing the request, and extracting parameter information in a request head and a request body from the request;

S403, dividing the request into a common request or a high concurrency request according to the parameter information through a preset classification rule.

In this embodiment, after the request control module is started in the control layer, continuous monitoring is performed through the network port or a specific API interface. The purpose of snooping is to receive requests from different application components in real time and to process requests from different sources to ensure consistency of service.

After receiving the request of the application component, the request control module needs to analyze the content of the request. The request typically includes a request header (e.g., request type, request source, user information, etc.) and a request body (e.g., specific query parameters or data manipulation content). The purpose of parsing is to extract key information in the request to determine the type of request.

The request control module judges and classifies the request according to a preset classification rule. Classification rules may be based on the content of the request, the source of the request, the amount of data requested, priority, etc. Through the judgment of these rules, the module can divide the request into a "normal request" or a "high concurrency request".

The classification rules are configured in advance and adjusted according to the actual requirements of the system. Rules may include conditional decisions on the type of request, amount of data, source, etc. to ensure accuracy of classification. A high concurrency request is typically determined based on a fixed condition or threshold, for example, when the amount of data requested exceeds a certain threshold.

When a request control module in the control layer is started, the system is configured with a monitored network port or an open API interface. The listening service may continue to run to receive requests from various application components in real-time. After receiving a request, the request control module analyzes the request. The parse operation includes reading basic information in the request header, such as request type, data size, request source, etc., and parsing specific operating parameters in the request body. The extraction of this information is the basis for the subsequent classification. And the module judges whether the request belongs to a common request or a high concurrency request by calling a preset classification rule according to the analyzed request information. The basis for classification may be the amount of data requested, the complexity of the operation, the priority of the source user, etc. According to the classification result, the request control module selects different processing paths, the common request is left at the control layer for processing, and the high concurrency request is forwarded to the data layer for processing.

According to the embodiment, the request is monitored and received in the control layer, and the request is analyzed and classified, so that the optimal processing path can be effectively selected according to the characteristics of the request. The request control module can flexibly judge the request type and schedule according to a preset classification rule, so that the utilization rate of system resources is improved, and the processing pressure in a high concurrency scene is reduced. Meanwhile, the module supports dynamic adjustment of classification rules, and ensures adaptability and flexibility under different service scenes.

In one embodiment, in S50, if the request is a normal request, accessing the target information from the distributed cache module according to the control connection parameter through the access interface provided by the request control module of the control layer includes:

s501, pre-configuring access interfaces through the request control module, and defining specific functions and applicable scenes of each access interface;

s502, determining a target access interface based on the type of the common request and combining the specific function and the applicable scene of each access interface;

s503, according to the control connection parameters, acquiring target information from the distributed cache module by calling the target access interface.

In this embodiment, in the request control module of the control layer, a plurality of standardized access interfaces are configured in advance. Each access interface is designed according to different functions and application scenarios, such as an account balance query interface, a transaction record query interface, and the like. The configuration of the interfaces includes setting the function of each interface, the type of request supported, processing logic, etc.

After receiving the ordinary request, the request control module dynamically selects an access interface most suitable for processing the ordinary request according to the type of the request (such as query type, writing type and the like) and the applicable scene of each access interface. This choice depends on a number of dimensions including the functional requirements of the request, the amount of data access, the response time, etc.

After the request control module selects a proper target access interface, the request control module can establish connection with the distributed cache module by calling the target access interface according to control connection parameters (such as the size of a connection pool, timeout time, retry times and the like) to acquire target information. The control connection parameters play a role in the process of stably and efficiently managing the connection.

When the control module is requested to start, a plurality of access interfaces are predefined and configured through a management platform or a configuration file. The configuration of each access interface includes the functional definition of the interface, the type of request supported, and the applicable scenario. For example, one interface may be defined as an "account information query interface" that is suitable for use in a fast query scenario for account data. After the request control module receives the common request and completes the preliminary analysis, the module can select a proper access interface according to the type of the request (such as reading, inquiring and the like) and combining the specific function and the applicable scene of each interface. The module may do this by querying an interface registry or consulting a preset interface mapping. After the target interface is determined, the request control module invokes the interface and references the control connection parameters to manage the connection resources. The control connection parameters include the size of the connection pool, timeout time, etc. The module establishes connection with the distributed cache module through the calling interface, and performs data query or other operations according to the functional logic of the interface. And the request control module queries the target information from the distributed cache module by calling the target access interface. After the information is successfully acquired, the module returns the data to the application component which initiates the request according to a preset path.

According to the method and the device, the access interfaces are preconfigured, and the functions and the applicable scenes of each interface are defined, so that the processing efficiency of the system on the common requests can be remarkably improved. By dynamically selecting a proper target access interface, the system can flexibly adapt to different request types and scenes and optimize the data access path. And in combination with management of control connection parameters, the access interface can be called to realize quick data access and response on the premise of keeping stability.

In one embodiment, the step S10 includes:

S101, deploying the data layer on a group of independent physical servers or virtual machines;

s102, deploying the control layer on another independent group of physical servers or virtual machines;

S103, configuring network connection between the data layer and the control layer.

In this embodiment, the main function of the data layer is to process data access requests with high concurrency, and perform operations related to data storage and caching. To ensure efficient operation of the data layer, the data layer is deployed separately on a separate physical server or virtual machine. The purpose of this is to improve the specificity of the computing resources and reduce the delay of data processing.

The control layer is used for receiving and analyzing the requests from the application components and scheduling and processing according to the request types. The deployment of the control layer is independent of the data layer and is mainly responsible for the classification and scheduling of requests. By independently deploying the control layers, computing tasks of request scheduling and data processing can be effectively isolated, and the expansibility of the system is improved.

Since the data layer and the control layer are deployed on different physical servers or virtual machines, efficient network connections need to be established to ensure the speed and reliability of communication between the data layer and the control layer. The configuration of the network connection may include selecting an appropriate network protocol, bandwidth and delay optimization strategy, etc.

The system, upon initialization, deploys the services of the data layer on a set of physical servers or virtual machines. Deployment of the data layer may be accomplished through an automated deployment tool (e.g., ansible, kubernetes, etc.) to ensure high availability and consistency of services. After deployment is complete, the data layer will focus on handling high concurrency requests from the control layer. The system, upon initialization, deploys the control layer's services on another set of physical servers or virtual machines. Deployment of the control layer may also be accomplished by an automated deployment tool. After deployment is complete, the control layer will focus on receiving and parsing requests from the various application components, and classifying and scheduling them. The system needs to ensure that the network connection between the data layer and the control layer is smooth during configuration. High-speed network links and optimized network protocols (e.g., TCP/IP, gRPC, etc.) are typically employed to ensure efficient transmission of data. Connection timeout and retry policies may also be set in the network configuration to ensure reliable communication between the data layer and the control layer.

According to the embodiment, the data processing architecture is divided into the data layer and the control layer, so that the resource utilization rate and expansibility of the system can be remarkably improved. The data layer is focused on high concurrency data processing, the control layer is responsible for classifying and scheduling requests, and the hierarchical architecture can effectively reduce the coupling of the system and improve the independence and flexibility of each layer. In addition, by the deployment of independent physical servers or virtual machines, the computing resources of the data layer and the control layer can be dynamically expanded according to actual service requirements, and the processing capacity of the system is improved.

In an embodiment, a data processing apparatus based on a hierarchical architecture is provided, where the data processing apparatus based on a hierarchical architecture corresponds to the data processing method based on a hierarchical architecture in the foregoing embodiment one by one. With reference to fig. 3, fig. 3 is a schematic functional block diagram of a data processing apparatus according to a preferred embodiment of the present invention. Architecture management module 10, concurrent data processing module 20, request control module 30, request identification module 40, generic request processing module 50, and concurrent data processing module 60. The functional modules are described in detail as follows:

an architecture management module 10 for dividing the data processing architecture into a data layer and a control layer;

The concurrent data processing module 20 is configured to deploy a concurrent data processing module in the data layer, and configure concurrent connection parameters of the concurrent data processing module and the distributed cache module;

a request control module 30, configured to deploy a request control module in the control layer, and configure control connection parameters of the request control module and the distributed cache module;

a request identification module 40, configured to receive a request from an application component in the control layer, and determine a type of the request, where the type of the request includes a normal request or a high concurrency request;

The general request processing module 50 is configured to access, if the request is a general request, target information from the distributed cache module according to the control connection parameter through an access interface provided by a request control module of the control layer, and return the target information to an application component that initiates the request;

And the concurrency data processing module 60 is configured to send the high concurrency request to the data layer if the request is a high concurrency request, access, through the concurrency data processing module of the data layer, target information from the distributed cache module according to the concurrency connection parameter, and return the target information to an application component that initiates the request.

In one embodiment, the concurrent data processing module 60 is specifically configured to:

When the concurrent data processing module fails to access the target information from the distributed cache module, acquiring the target information from a database through the concurrent data processing module;

Caching the target information into the distributed caching module;

And returning the target information to the application component which initiates the request.

In one embodiment, the concurrent data processing module 20 is specifically configured to:

installing the concurrent data processing module on a physical server or virtual machine of the data layer;

setting concurrent connection parameters with the distributed cache module in the concurrent data processing module, wherein the concurrent connection parameters comprise a connection pool size, timeout time and retry times;

initializing a connection pool according to the size of the connection pool, and pre-distributing available connection with the distributed cache module in the connection pool.

Determining idle connection from a pre-initialized connection pool through the concurrent data processing module, and executing data access operation in the distributed cache module through the idle connection;

Monitoring the running state of the current connection through the concurrent data processing module, and judging whether the current connection has connection overtime according to the overtime time in the concurrent connection parameters;

if the concurrent data processing module detects that the current connection is overtime, releasing the current connection, and reestablishing the connection with the distributed cache module according to the retry times in the concurrent connection parameters;

And when the target information is successfully accessed, releasing and returning the used current connection to the connection pool through the concurrent data processing module.

In one embodiment, the request identification module 40 is specifically configured to:

monitoring and receiving a request from an application component through a request control module in the control layer;

analyzing the request, and extracting parameter information in a request head and a request body from the request;

And dividing the request into a common request or a high concurrency request according to the parameter information through a preset classification rule.

In one embodiment, the general request processing module 50 is specifically configured to:

The access interfaces are preconfigured through the request control module, and specific functions and applicable scenes of each access interface are defined;

Determining a target access interface based on the type of the common request and combining the specific function and the applicable scene of each access interface;

and according to the control connection parameters, acquiring target information from the distributed cache module by calling the target access interface.

In one embodiment, the architecture management module 10 is specifically configured to:

deploying the data layer on a set of independent physical servers or virtual machines;

deploying the control layer on another set of independent physical servers or virtual machines;

and configuring network connection between the data layer and the control layer.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 4. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes non-volatile and/or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external user terminal through network connection. The computer program, when executed by a processor, performs functions or steps of a server side of a data processing method based on a hierarchical architecture.

In one embodiment, a computer device is provided, which may be a user terminal, and the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is for communicating with an external server via a network connection. The computer program, when executed by the processor, implements functions or steps of a client side of a data processing method based on a hierarchical architecture

In one embodiment, a computer device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:

dividing a data processing architecture into a data layer and a control layer;

In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:

dividing a data processing architecture into a data layer and a control layer;

It should be noted that, the functions or steps that can be implemented by the computer readable storage medium or the computer device may correspond to the descriptions of the server side and the client side in the foregoing method embodiments, and are not described herein for avoiding repetition.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.

It should be noted that, if a software tool or component other than the company appears in the embodiment of the present application, the embodiment is merely presented by way of example, and does not represent actual use. The foregoing embodiments are merely illustrative of the technical solutions of the present application, and not restrictive, and although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that modifications may still be made to the technical solutions described in the foregoing embodiments or equivalent substitutions of some technical features thereof, and that such modifications or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. The data processing method based on the layered architecture is characterized by comprising the following steps of:

dividing a data processing architecture into a data layer and a control layer;

2. The data processing method based on a layered architecture according to claim 1, further comprising, after accessing, by the concurrent data processing module of the data layer, the target information from the distributed cache module according to the concurrent connection parameter:

Caching the target information into the distributed caching module;

3. The data processing method based on the layered architecture according to claim 1, wherein deploying a concurrent data processing module in the data layer and configuring concurrent connection parameters of the concurrent data processing module and a distributed cache module includes:

4. The data processing method based on the layered architecture according to claim 1, wherein accessing, by the concurrent data processing module of the data layer, the target information from the distributed cache module according to the concurrent connection parameter comprises:

5. A data processing method based on a hierarchical architecture as claimed in claim 1, wherein in said control layer, a request from an application component is received and a type of said request is determined, said type of request comprising a normal request or a high concurrency request, comprising:

6. The data processing method based on a layered architecture according to claim 1, wherein if the request is a normal request, accessing the target information from the distributed cache module according to the control connection parameter through an access interface provided by a request control module of the control layer, comprises:

7. The hierarchical architecture-based data processing method of claim 1, wherein dividing the data processing architecture into a data layer and a control layer comprises:

8. A data processing apparatus based on a layered architecture, the data processing apparatus based on a layered architecture comprising:

9. A computer device comprising a memory, a processor and a hierarchical-architecture-based data processing program stored onto the memory and executable on the processor, the hierarchical-architecture-based data processing program implementing the steps of the hierarchical-architecture-based data processing method according to any one of claims 1-7 when executed by the processor.

10. A computer readable storage medium, characterized in that the storage medium has stored thereon a hierarchical architecture based data processing program, which when executed by a processor, implements the steps of the hierarchical architecture based data processing method according to any of claims 1-7.