CN112738060A - Method and device for processing micro-service data, micro-service processing platform and medium - Google Patents
Method and device for processing micro-service data, micro-service processing platform and medium Download PDFInfo
- Publication number
- CN112738060A CN112738060A CN202011556562.6A CN202011556562A CN112738060A CN 112738060 A CN112738060 A CN 112738060A CN 202011556562 A CN202011556562 A CN 202011556562A CN 112738060 A CN112738060 A CN 112738060A
- Authority
- CN
- China
- Prior art keywords
- micro
- service
- data
- microservice
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 162
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000006978 adaptation Effects 0.000 claims abstract description 22
- 238000012544 monitoring process Methods 0.000 claims description 71
- 238000004458 analytical method Methods 0.000 claims description 44
- 238000004590 computer program Methods 0.000 claims description 15
- 230000010354 integration Effects 0.000 claims description 8
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 238000004140 cleaning Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000003672 processing method Methods 0.000 abstract description 4
- 238000012827 research and development Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1073—Registration or de-registration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Multimedia (AREA)
- Debugging And Monitoring (AREA)
- Stored Programmes (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application is applicable to the technical field of research and development management, and provides a micro-service data processing method, a device, a micro-service processing platform and a medium, wherein the method comprises the following steps: determining a target registration center containing registration information of the micro service and a target registration type of the target registration center; generating a target calling interface in a micro-service registration adaptation engine of a micro-service processing platform according to the target registration type; acquiring micro-service data of the micro-service according to the target calling interface and the target open interface; and writing the micro-service data into a database of the micro-service processing platform, and performing centralized processing. By adopting the method, the interface adaptation is carried out with the registration centers of different registration types to obtain the micro-service data of each micro-service, and the effect of directly and effectively processing the micro-service data can be realized when the registration information of a plurality of micro-services is respectively deployed in the registration centers of different registration types.
Description
Technical Field
The application belongs to the technical field of big data, and particularly relates to a micro-service data processing method and device, a micro-service processing platform and a medium.
Background
In the prior art, when registration information of a plurality of micro services is respectively deployed in registration centers of different registration types, it is difficult to directly process the plurality of micro services through a single micro service processing platform due to the different registration types of the registration centers. At present, registration information of a large number of micro services is usually deployed in a registry of the same registration type. However, if a large number of micro services have corresponding registration centers, the registration information deployed by the micro services in the corresponding registration centers needs to be deleted. And then, the registration information is redeployed in the same registration type registration center. At this time, when the registration information is redeployed, the workload of the worker is increased. Therefore, at present, when registration information of a plurality of micro services are respectively deployed in registration centers of different registration types, it is difficult to directly and effectively process a plurality of micro service data through a single micro service processing platform.
Disclosure of Invention
The embodiment of the application provides a method and a device for processing micro-service data, a micro-service processing platform and a medium, which can solve the problem that when registration information of a plurality of micro-services is respectively deployed in registration centers with different registration types, the plurality of micro-service data are difficult to be directly and effectively processed through a single micro-service processing platform.
In a first aspect, an embodiment of the present application provides a method for processing microservice data, which is applied to a microservice processing platform, and the method includes:
determining a target registration center containing registration information of the micro service, and determining a target registration type of the target registration center;
generating a target calling interface adapted to a target open interface of the target registration center in a micro-service registration adaptation engine of the micro-service processing platform according to the target registration type;
acquiring micro-service data of the micro-service according to the target calling interface and the target open interface;
and writing the micro-service data into a database of the micro-service processing platform, and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data corresponding to a plurality of micro-services one by one.
In one embodiment, the target registry comprises a unified registry of the microservice processing platform and an existing registry of the microservice; before the determining the target registry containing the registration information of the microservice, the method further comprises the following steps:
if the micro service does not have a corresponding registration center, acquiring registration information of the micro service, and deploying the registration information in the unified registration center;
and if the micro service has a corresponding registration center, determining the registration center corresponding to the micro service as a target registration center of the micro service.
In one embodiment, before the determining the target registry containing the registration information of the microservice, the method further comprises:
if the micro service has a corresponding registration center, deleting the registration information deployed by the micro service in the corresponding registration center;
and acquiring the registration information of the micro service, and deploying the registration information in the unified registration center again.
In an embodiment, after the writing the micro service data into the database of the micro service processing platform and performing centralized processing on the micro service data, the method further includes:
determining a frame type of a microservice frame used by the microservice;
calling a tool kit matched with the frame type through a micro-service monitoring engine of the micro-service processing platform;
and when the micro service is operated, accessing a micro service framework of the micro service according to the tool kit so as to call a service component of the micro service.
In an embodiment, when the microservice is run, after accessing a microservice framework of the microservice according to the toolkit to call a service component of the microservice, the method further includes:
acquiring monitoring data of the micro-service during operation;
performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result;
storing the analysis results in a database.
In an embodiment, the acquiring the monitoring data of the microservice runtime includes:
storing the monitoring data of the micro service in operation into a message queue of the micro service;
and writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
In an embodiment, the performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result, and obtaining the analysis result includes:
sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain processed monitoring data;
integrating the processed monitoring data in a charting mode to respectively obtain table data in a table mode and graphic data in a graphic mode;
and analyzing by combining the table data and the graph data to obtain the analysis result.
In a second aspect, an embodiment of the present application provides an apparatus for processing microservice data, which is applied to a microservice processing platform, and the apparatus includes:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for determining a target registration center containing registration information of the micro service and determining a target registration type of the target registration center;
the generating module is used for generating a target calling interface which is adaptive to a target open interface of the target registration center in a micro service registration adaptive engine of the micro service processing platform according to the target registration type;
the second acquisition module is used for acquiring the micro-service data of the micro-service according to the target calling interface and the target open interface;
and the processing module is used for writing the micro-service data into a database of the micro-service processing platform and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data which correspond to a plurality of micro-services one by one.
In a third aspect, an embodiment of the present application provides a microservice processing platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the computer program, implements the method according to any one of the above first aspects.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the method according to any one of the above first aspects.
In a fifth aspect, embodiments of the present application provide a computer program product, which, when run on a terminal device, causes the terminal device to perform the method of any one of the above aspects.
Compared with the prior art, the embodiment of the application has the advantages that: after determining the target registration center corresponding to the micro-service and the target registration type of the target registration center, the micro-service processing platform only needs to adapt to the target open interface of the existing target registration center. Therefore, the micro-service processing platform can be connected with the target registration center, and the registration information deployed by the micro-service in the target registration center can be acquired. And then, acquiring the micro service data of the corresponding micro service according to the registration information for processing. Based on the method, the micro-service processing platform can not only uniformly process the micro-service data of a plurality of micro-services corresponding to the registration centers of different registration types, but also effectively reduce the workload of workers and reduce the risk of micro-service data leakage when the registration information is redeployed in the micro-service processing platform.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a flowchart illustrating an implementation of a method for processing microservice data according to an embodiment of the present application;
fig. 2 is a flowchart of an implementation of a method for processing microservice data according to another embodiment of the present application;
fig. 3 is a flowchart illustrating an implementation of a method for processing microservice data according to yet another embodiment of the present application;
fig. 4 is a flowchart of an implementation of a method for processing microservice data according to another embodiment of the present application;
fig. 5 is a flowchart of an implementation of a method for processing microservice data according to another embodiment of the present application;
fig. 6 is a schematic diagram illustrating an implementation manner of S10CA of a method for processing microservice data according to an embodiment of the present application;
fig. 7 is a schematic diagram illustrating an implementation manner of S10CB of a method for processing microservice data according to an embodiment of the present application;
fig. 8 is a block diagram illustrating a micro-service data processing apparatus according to an embodiment of the present application;
fig. 9 is a block diagram illustrating a micro-service processing platform according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
The method for processing the micro-service data provided by the embodiment of the application is applied to a micro-service processing platform. The micro-service processing platform comprises a unified registration center, a micro-service registration adaptation engine, a micro-service monitoring engine and a database. The unified registry can be used for deploying the registration information of each micro-service, so that each newly online micro-service can realize the centralized processing of the registration information of the micro-services without deploying the respective registration information in a plurality of registries. The microservice registration adaptation engine is used for carrying out interface adaptation on the existing registration center of the microservice so that the microservice processing platform can acquire registration information of the microservice from the existing registration center through the interface and acquire microservice data of the microservice according to the registration information. The microservice monitoring engine can be used for calling a tool kit matched with the frame type to access the microservice frame after the frame type of the microservice frame in the microservice is determined so as to call microservice components in the microservice. The database may be used to store micro-service data of each micro-service, monitoring data generated during the operation of the micro-service, and an analysis result obtained by analyzing the monitoring data, which is not limited herein.
Referring to fig. 1, fig. 1 is a flowchart illustrating an implementation of a method for processing microservice data according to an embodiment of the present application, where the method is applied to a microservice processing platform, and the method specifically includes the following steps:
s101, determining a target registration center containing registration information of the micro-service, and determining a target registration type of the target registration center.
In application, the micro-service is an architecture scheme for constructing service applications. The architecture scheme of the service application is different from the traditional monolithic scheme, and the service application can be split into a plurality of core service functions. Each service function may be referred to as a service and may be built and deployed separately, i.e., each service function may be considered a microservice. When each micro service is started, data information such as a network address of the micro service, a micro service component and the like needs to be registered in a registration center and stored by the registration center. Based on this, when invoking the microservice, the service consumer may query the network address of the microservice from the registry. And then, acquiring the micro-service component through the network address, connecting the micro-service component with a calling interface for accessing the micro-service, and calling the micro-service component.
In application, whether the micro service has the corresponding target registration center and the target registration type of the target registration center is determined, and the determination result is sent to the micro service processing platform according to the actual situation.
In application, the registration type of the above-mentioned registry is various, and specifically, the registration type includes, but is not limited to, Zookeeper type, Eureka type, and the like. It will be appreciated that the registry may be used with the same or different types of registrations for different microservices. It can be understood that the registration types of the micro services corresponding to the registries are different, and it is difficult for the micro service processing platform of each micro service to be compatible with the registries of other registration types. Further, it is difficult to implement uniform processing for micro-services corresponding to a plurality of different registration types.
S102, according to the target registration type, generating a target calling interface adapted to a target open interface of the target registration center in a micro-service registration adaptation engine of the micro-service processing platform.
In the application, it has been described above that when each micro service is started, data information such as a network address and a micro service component of the first micro service needs to be deployed in the registry. The data information such as network addresses, micro-service components and the like correspondingly deployed by all micro-services in the target registration center can be obtained after the micro-service registration adaptation engine is in adaptation connection with the target registration center. Based on this, in order to realize the adaptive connection between the micro-service registration adaptation engine and the registration center, an open interface of the target registration center needs to be acquired. And then, the micro-service registration adaptation engine adapts the target registration center according to the open interface to generate a target calling interface which can carry out data transmission with the target registration center. The calling interface and the open interface can be API interfaces. It should be noted that the microservice registration adaptation engine is a component in the microservice processing platform, and is used for adapting open interfaces provided by registries of different registration types and generating a call interface corresponding to the registry of each registration type.
It will be appreciated that a component (registry) has an open API, i.e. data itself (data within the registry) can be opened to the outside on behalf of the component. In this case, the open API includes usage rights of the API, for example, the number of calls, call objects, and the like of the API.
S103, acquiring micro service data of the micro service according to the target calling interface and the target open interface.
And S104, writing the micro-service data into a database of the micro-service processing platform, and performing centralized processing on the micro-service data, wherein the database contains micro-service data corresponding to a plurality of micro-services one by one.
In application, the microservice data may be understood as data information such as a network address of a microservice, a microservice component, and the like. It should be noted that, when the target call interface of the micro-service registration adaptation engine is adapted to the target open interface of the target registration center, the micro-service processing platform may call the target call interface to connect with the target open interface. Further, the network address of the micro-service and the data information such as the micro-service component can be searched in the target registry.
In application, after the micro-service data is obtained, the micro-service data is written into a database of the micro-service processing platform for storage. Therefore, after the database contains the micro-service data which are in one-to-one correspondence with the plurality of micro-services, the centralized processing of the plurality of micro-service data can be realized. The centralized processing may be to set identification information of each micro service data to distinguish each micro service, and may perform processing such as adding, deleting, modifying, querying, and the like on registration information deployed by each micro service in the target registration center, which is not limited herein.
In this embodiment, after determining the target registration center corresponding to the micro service and the target registration type of the target registration center, the micro service processing platform only needs to adapt to a target open interface of an existing target registration center. Therefore, the micro-service processing platform can be connected with the target registration center, and the registration information deployed by the micro-service in the target registration center can be acquired. And then, acquiring the micro service data of the corresponding micro service according to the registration information for processing. Based on the method, the micro-service processing platform can not only uniformly process the micro-service data of a plurality of micro-services corresponding to the registration centers of different registration types, but also effectively reduce the workload of workers and reduce the risk of micro-service data leakage when the registration information is redeployed in the micro-service processing platform.
Referring to fig. 2, in an embodiment, the target registry includes a unified registry of the microservice processing platform and a registry already existing for the microservice; before determining the target registry containing the registration information of the microservice at S101, the following steps S101A-S101B are further included, which are detailed as follows:
S101A, if the micro service does not have a corresponding registration center, acquiring the registration information of the micro service, and deploying the registration information in the unified registration center.
In application, the unified registry may be a registry which is set in advance in the microservice processing platform for a worker, and may be used to deploy registration information for microservices which do not have a corresponding registry.
In an application, for newly brought-on-line microservices, they typically do not have a corresponding registry. Based on the method, the registration information of the micro-service can be deployed in a unified registration center in the micro-service processing platform. At this time, there is only one unified registry, that is, there is only one registration type of the unified registry. At this time, the unified registry is the target registry, and the unified registry and the micro-service registration adaptation engine are all components in the micro-service processing platform. Namely, the microservice registration adaptation engine is considered to have a target calling interface adapted to the target open interface of the unified registry. Therefore, the registration information of the newly online micro-service is deployed in the unified registration center, the registration center types corresponding to the newly online micro-service can be unified, the registration centers with various registration types are prevented from being constructed in the micro-service processing platform, and the component resources required to be used in the micro-service processing platform are reduced.
S101B, if the micro service has a corresponding registration center, determining the registration center corresponding to the micro service as a target registration center of the micro service.
In application, for the micro-service of the existing corresponding registration center, at this time, the existing corresponding registration center can be determined to be the target registration center. Based on this, the way of S101 to S102 described above can be adopted, and the microservice registration adaptation engine is used to perform interface adaptation on the target open interface of the target registry.
In this embodiment, after determining that the micro service has a corresponding registration center, the existing registration center is subjected to interface adaptation by using the above S101 to S102 to obtain micro service data of the micro service, so that the registration information of the micro service does not need to be redeployed, thereby reducing the workload of the worker; and after determining that the micro-service has no corresponding registration center, deploying the registration information of each micro-service by using the uniform registration center of the same registration type, so that the registration centers of various registration types can be prevented from being constructed in the micro-service processing platform, and the component resources required to be used in the micro-service processing platform are reduced.
Referring to FIG. 3, in one embodiment, before determining the target registry containing the registration information of the microservice at S101, the following steps S101C-S101D are further included, which are detailed as follows:
S101C, if the micro service has a corresponding registration center, deleting the registration information deployed by the micro service in the corresponding registration center.
S101D, obtaining the registration information of the micro service, and re-deploying the registration information in the unified registry.
In application, after determining that the micro service has the corresponding registration center, the registration information deployed by the micro service in the corresponding registration center can be deleted. And then, the registration information of the microservice is deployed in the unified registration center again. It should be noted that, when the number of the micro services of the existing corresponding registry is small, the registration information may be redeployed by adopting the above manner, so as to reduce the number of interfaces for the micro service registration adaptation engine to perform interface adaptation with the external registry.
Referring to fig. 4, in an embodiment, after the step S104 writes the micro service data into the database of the micro service processing platform and performs centralized processing on the micro service data, the following steps S104A-S104C are further included, which are detailed as follows:
S104A, determining the frame type of the micro service frame used by the micro service.
In an application, the micro-service framework can be used for deploying a running environment required by the micro-service in running. Therefore, when the micro service is generated, a micro service framework of the micro service needs to be constructed in advance to deploy various service functions of the micro service in the micro service framework so that the micro service can operate normally. The types of the framework of the microservice framework used are different because the service functions are different between different microservices. The frame type includes, but is not limited to, a Service Mesh frame type, a Linkerd frame type, and the like.
It should be noted that, for determining the framework type of the microservice framework, the method for determining the target registration type of the target registry may be consistent. For example, the staff member may determine the micro service according to the actual situation of the micro service, and send the determination result to the micro service processing platform, which is not limited herein.
S104B, calling the tool kit adapted to the frame type through the micro service monitoring engine of the micro service processing platform.
S104C, when the micro service is operated, accessing a micro service framework of the micro service according to the tool kit to call a service component of the micro service.
In an application, after acquiring micro-service data of a micro-service, in order to run the micro-service (i.e. call a micro-service component in the micro-service data), a corresponding micro-service framework needs to be deployed in a micro-service processing platform. Based on the above, in order to avoid deploying the micro-service framework of each framework type in the micro-service processing platform, the micro-service processing platform can call the tool kit adapted to the framework type and the micro-service framework to connect through the micro-service monitoring engine. Furthermore, the micro-service processing platform can be remotely accessed to the micro-service framework, so that the micro-service components can be called. The microservice monitoring engine is a functional component in the microservice processing platform, is used for storing tool packages respectively matched with a plurality of frame types, and can be used for matching with microservice frames of corresponding frame types by calling the tool packages.
In an application, the above-mentioned tool package is a Software Development Kit (SDK), and a set of Development tools for application Software is generally established for a specific Software package, a Software framework (micro service framework), an operating system, and the like. That is, the staff member can install a tool kit adapted to various frame types in the microservice monitoring engine in advance, so that the microservice processing platform can access the microservice frame.
In the prior art, micro services of different framework types generally have respective corresponding micro service processing platforms. In addition, because the frame types are different, a plurality of micro service processing platforms are not mutually adaptive. Based on the method, the SDK toolkit can be used for adapting the capability of the micro-services with multiple framework types and accessing the micro-service frameworks of the micro-services, so that the micro-services with different framework types can be called and processed.
Referring to fig. 5, in an embodiment, after accessing the micro service framework of the micro service according to the toolkit to call the service component of the micro service when the micro service is executed at S104C, the following steps S104CA-S104CC are further included, which are detailed as follows:
and S104CA, acquiring the monitoring data of the micro service during operation.
In application, after the micro-service processing platform is accessed to the micro-service framework through the SDK toolkit, a micro-service component of the micro-service can be called, and then the micro-service is operated. Based on this, when the micro-service is running, the micro-service processing platform can obtain the monitoring data when the micro-service is running. The monitoring data includes, but is not limited to, the working time of the microservice, processed processing data, and the like.
And S104CB, performing real-time integration analysis on the monitoring data in a preset time period to obtain an analysis result.
S104CC, storing the analysis result in a database.
In application, the time period may be a period preset by a user. The micro-service processing platform can integrate the monitoring data in each time period and then analyze the monitoring data to obtain an analysis result. The analysis result may be the result of the micro-service in the time period, such as the working efficiency and the working state. By analyzing the running state of the micro-service in a certain time period, the working personnel can reasonably adjust and optimize the micro-service configuration of the micro-service better according to the analysis result of the micro-service. The microservice configuration includes, but is not limited to, the running environment of the microservice, and the functional components of the microservice.
In application, the database is used for storing analysis results, and can store the analysis results in a diagrammatized manner, so that the microservice processing platform can sort, plan and analyze the disordered analysis results (analyze again according to the analysis results of different time periods). Furthermore, the staff can adjust and optimize the configuration information of the micro-service in real time according to the existing analysis result in the database.
Referring to fig. 6, in an embodiment, in acquiring the monitoring data of the micro service runtime at S104CA, the following sub-steps S104CA1-S104CA2 are further included, which are detailed as follows:
and S104, the CA1 stores the monitoring data of the micro service operation into the message queue of the micro service.
And S104CA2, writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
In an application, the message queue is a temporary storage area in the microservice, and is used for storing all data (monitoring data) of the microservice in operation. The file system is a local file system in the micro-service processing platform, wherein the file system can persistently store the monitoring data. In the actual process, problems such as crash, restart and the like may occur when the micro service runs. Based on this, if the micro service crashes suddenly during operation, the monitoring data stored in the message queue may have a data loss condition during the process of recovering the operation of the micro service. Therefore, after storing data in a message queue local to the microservice, monitoring data in the message queue needs to be stored in the file system to prevent the monitoring data of the microservice from being lost.
In application, the thread refers to an execution flow in a process, and is used for executing a service in the microservice. Asynchronous is understood to mean a way of transmitting data. Specifically, when the monitoring data stored in the message queue is written into the file system by using the asynchronous thread, the monitoring data does not need to be synchronously processed between the micro-service processing platform and the micro-service. For example, the micro service stores the monitoring data to the message queue in real time, and the micro service processing platform may obtain the monitoring data in the time period from the message queue every preset time period and store the monitoring data to the file system. Therefore, the micro-service processing platform and the micro-service do not need to wait for the response of the other party, and the data transmission efficiency is improved. The file system is specifically an HDFS system (distributed file system). Because the data volume transmitted between the micro-service processing platform and the plurality of micro-services is very large, the HDFS system has the characteristic of high fault tolerance, and can provide high throughput to acquire the monitoring data in the plurality of micro-services.
Referring to fig. 7, in an embodiment, in step S104CB, the real-time integration analysis of the monitored data within the preset time period is performed to obtain an analysis result, and the analysis result further includes the following sub-steps S104CB1-S104CB3, which are detailed as follows:
and S104CB1, sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain the processed monitoring data.
In an application, the monitoring data is generally raw data of the micro service operation, and formats and dimensions of the raw data may be different. Based on this, data cleaning processing, format conversion processing, and the like are required for the monitoring data. The data cleaning process may be to clean data with transmission errors in the monitoring data. In an application, the obtained monitoring data may be monitoring data in multiple formats, such as double, int, char, and the like. And the data format of each monitoring data can be unified for the convenience of subsequent analysis and processing. That is, the current data format of the monitoring data is converted into the data in the preset target format, which is not limited in this respect. At this time, the data in the target format is the processed monitoring data.
S104CB2, integrating the processed monitoring data in a charting mode to respectively obtain table data in a table mode and graph data in a graph mode.
And S104CB3, analyzing by combining the table data and the graph data to obtain the analysis result.
In application, the data integration is performed on the processed monitoring data, and the data of the same type can be integrated. For example, the processed monitoring data includes data of the micro-service working efficiency and some type of data processed by the micro-service. Based on this, integrating the processed monitoring data can be understood as centralizing and analyzing all data belonging to the micro-service working efficiency.
In application, the analysis result obtained by analyzing the table data may be obtained by performing operations (difference operation, square error operation) and the like with each data value in the previous table data directly according to each data value in the table data, so as to obtain an operation numerical value. And then, determining a target numerical range containing the operation numerical value according to a plurality of preset numerical ranges and the analysis result corresponding to each numerical range, and obtaining the analysis result of the table data. The analysis result obtained by analyzing the image data may be that the image data is compared with a plurality of preset images, and a target preset image with the highest similarity to the image data is determined from the preset images. Based on the analysis result, the analysis result corresponding to the target preset graph can be determined from the analysis results corresponding to the preset graphs respectively.
In application, after the analysis result is obtained, the micro service processing platform monitoring data can also display table data in a table form and graphic data in a graphic form. It can be understood that the table data can only visually show the data values, and the relation between the data values cannot be clearly shown. Based on the method, the graphic data in the graphic form are displayed simultaneously, so that when a user sees the table data, the user can understand the internal rules of the table data more easily according to the graphic data, and the user can understand the monitoring data more comprehensively.
In application, the micro-service processing platform can also receive a control instruction issued by a worker, so that the micro-service processing platform can manage the working state of the micro-service according to the control instruction. Specifically, the control instruction includes, but is not limited to, an instruction for controlling the microserver to change the execution task, and an instruction for controlling the microserver to stop executing the task. The operating state of the control microservice may be a service state, an idle state, or the like. It is necessary to supplement that, when the working state of the micro-service changes, it can also save the latest working state of the micro-service in the database by the data transmission mode of the asynchronous thread.
Referring to fig. 8, fig. 8 is a block diagram illustrating a micro service data processing apparatus according to an embodiment of the present disclosure. The processing apparatus of microservice data in this embodiment includes modules for executing the steps in the embodiments corresponding to fig. 1 to fig. 7. Please refer to fig. 1 to 7 and fig. 1 to 7 for related descriptions. For convenience of explanation, only the portions related to the present embodiment are shown. Referring to fig. 8, a device 800 for processing microservice data includes: a first obtaining module 810, a generating module 820, a second obtaining module 830, and a processing module 840, wherein:
the first obtaining module 810 is configured to determine a target registry including registration information of the microservice, and determine a target registration type of the target registry.
A generating module 820, configured to generate, in the micro service registration adaptation engine of the micro service processing platform, a target call interface adapted to the target open interface of the target registration center according to the target registration type.
A second obtaining module 830, configured to obtain the microservice data of the microservice according to the target calling interface and the target open interface.
The processing module 840 is configured to write the micro service data into a database of the micro service processing platform, and perform centralized processing on the micro service data, where the database includes micro service data corresponding to multiple micro services one to one.
In one embodiment, the target registry comprises a unified registry of the microservice processing platform and an existing registry of the microservice; the apparatus 800 for processing microservice data further comprises:
and the third acquisition module is used for acquiring the registration information of the micro service and deploying the registration information in the unified registration center if the micro service does not have the corresponding registration center.
And the first determining module is used for determining the registry corresponding to the micro service as a target registry of the micro service if the micro service has the corresponding registry.
In an embodiment, the apparatus 800 for processing microservice data further includes:
and the deleting module is used for deleting the registration information deployed by the micro service in the existing corresponding registration center if the micro service has the corresponding registration center.
And the deployment module is used for acquiring the registration information of the micro-service and re-deploying the registration information in the unified registration center.
In an embodiment, the apparatus 800 for processing microservice data further includes:
a second determining module for determining a frame type of a microservice frame used by the microservice.
And the first calling module is used for calling the tool kit matched with the frame type through a micro-service monitoring engine of the micro-service processing platform.
And the second calling module is used for accessing a micro-service framework of the micro-service according to the tool kit when the micro-service is operated so as to call a service component of the micro-service.
In an embodiment, the apparatus 800 for processing microservice data further includes:
and the third acquisition module is used for acquiring the monitoring data during the micro-service operation.
And the analysis module is used for performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result.
And the storage module is used for storing the analysis result into a database.
In an embodiment, the third obtaining module is further configured to:
storing the monitoring data of the micro service in operation into a message queue of the micro service;
and writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
In one embodiment, the analysis module is further configured to:
sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain processed monitoring data;
integrating the processed monitoring data in a charting mode to respectively obtain table data in a table mode and graphic data in a graphic mode;
and analyzing by combining the table data and the graph data to obtain the analysis result.
It should be understood that, in the structural block diagram of the data processing apparatus shown in fig. 8, each unit/module is used to execute each step in the embodiment corresponding to fig. 1 to 7, and each step in the embodiment corresponding to fig. 1 to 7 has been explained in detail in the above embodiment, and specific reference is made to the relevant description in the embodiment corresponding to fig. 1 to 7 and fig. 1 to 7, which is not repeated herein.
Fig. 9 is a block diagram of a microservice processing platform according to another embodiment of the present application. As shown in fig. 9, the microservice processing platform 900 of this embodiment includes: a processor 910, a memory 920 and a computer program 930, e.g. a program of a data processing method, stored in the memory 920 and executable on the processor 910. The processor 910 implements the steps in the embodiments of the data processing methods described above, such as S101 to S104 shown in fig. 1, when executing the computer program 930. Alternatively, the processor 910, when executing the computer program 930, implements the functions of the modules in the embodiment corresponding to fig. 8, for example, the functions of the modules 810 to 840 shown in fig. 8, please refer to the related description in the embodiment corresponding to fig. 8.
Illustratively, the computer program 930 may be divided into one or more units, which are stored in the memory 920 and executed by the processor 910 to accomplish the present application. One or more elements may be a sequence of computer program instruction segments capable of performing certain functions, which are used to describe the execution of computer program 930 in terminal device 900.
The microservice processing platform may include, but is not limited to, a processor 910, a memory 920. Those skilled in the art will appreciate that fig. 9 is merely an example of a microservice processing platform 900 and does not constitute a limitation of the microservice processing platform 900 and may include more or less components than those shown, or some components may be combined, or different components may be included, for example, the microservice processing platform may also include input output devices, network access devices, buses, etc.
The processor 910 may be a central processing unit, but may also be other general purpose processors, digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A method for processing microservice data is applied to a microservice processing platform, and the method comprises the following steps:
determining a target registration center containing registration information of the micro service, and determining a target registration type of the target registration center;
generating a target calling interface adapted to a target open interface of the target registration center in a micro-service registration adaptation engine of the micro-service processing platform according to the target registration type;
acquiring micro-service data of the micro-service according to the target calling interface and the target open interface;
and writing the micro-service data into a database of the micro-service processing platform, and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data corresponding to a plurality of micro-services one by one.
2. The method of claim 1, wherein the target registry comprises a unified registry of the microservice processing platform and a registry already owned by the microservice;
before the determining the target registry containing the registration information of the microservice, the method further comprises the following steps:
if the micro service does not have a corresponding registration center, acquiring registration information of the micro service, and deploying the registration information in the unified registration center;
and if the micro service has a corresponding registration center, determining the registration center corresponding to the micro service as a target registration center of the micro service.
3. The method of claim 2, wherein prior to determining the target registry containing registration information for the microservice, further comprising:
if the micro service has a corresponding registration center, deleting the registration information deployed by the micro service in the corresponding registration center;
and acquiring the registration information of the micro service, and deploying the registration information in the unified registration center again.
4. The method for processing microservice data according to claim 1, wherein after the writing of the microservice data into the database of the microservice processing platform and the centralized processing of the microservice data, the method further comprises:
determining a frame type of a microservice frame used by the microservice;
calling a tool kit matched with the frame type through a micro-service monitoring engine of the micro-service processing platform;
and when the micro service is operated, accessing a micro service framework of the micro service according to the tool kit so as to call a service component of the micro service.
5. The method for processing microservice data according to claim 4, wherein when the microservice is run, after accessing a microservice framework of the microservice according to the toolkit to invoke a service component of the microservice, the method further comprises:
acquiring monitoring data of the micro-service during operation;
performing real-time integration analysis on the monitoring data within a preset time period to obtain an analysis result;
storing the analysis results in a database.
6. The method for processing microservice data according to claim 5, wherein the obtaining the monitoring data of the microservice runtime includes:
storing the monitoring data of the micro service in operation into a message queue of the micro service;
and writing the monitoring data in the message queue into a file system of the micro-service processing platform through an asynchronous thread.
7. The method for processing microservice data according to claim 5, wherein the performing real-time integration analysis on the monitored data within a preset time period to obtain an analysis result comprises:
sequentially carrying out data cleaning processing and data format conversion processing on the monitoring data to obtain processed monitoring data;
integrating the processed monitoring data in a charting mode to respectively obtain table data in a table mode and graphic data in a graphic mode;
and analyzing by combining the table data and the graph data to obtain the analysis result.
8. An apparatus for processing microservice data, applied to a microservice processing platform, the apparatus comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for determining a target registration center containing registration information of the micro service and determining a target registration type of the target registration center;
the generating module is used for generating a target calling interface which is adaptive to a target open interface of the target registration center in a micro service registration adaptive engine of the micro service processing platform according to the target registration type;
the second acquisition module is used for acquiring the micro-service data of the micro-service according to the target calling interface and the target open interface;
and the processing module is used for writing the micro-service data into a database of the micro-service processing platform and carrying out centralized processing on the micro-service data, wherein the database contains micro-service data which correspond to a plurality of micro-services one by one.
9. A microservice processing platform comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011556562.6A CN112738060B (en) | 2020-12-24 | 2020-12-24 | Method and device for processing micro-service data, micro-service processing platform and medium |
| PCT/CN2021/084251 WO2022134358A1 (en) | 2020-12-24 | 2021-03-31 | Microservice data processing method, apparatus, microservice processing platform, and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011556562.6A CN112738060B (en) | 2020-12-24 | 2020-12-24 | Method and device for processing micro-service data, micro-service processing platform and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112738060A true CN112738060A (en) | 2021-04-30 |
| CN112738060B CN112738060B (en) | 2022-11-18 |
Family
ID=75615666
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011556562.6A Active CN112738060B (en) | 2020-12-24 | 2020-12-24 | Method and device for processing micro-service data, micro-service processing platform and medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112738060B (en) |
| WO (1) | WO2022134358A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113656096A (en) * | 2021-08-10 | 2021-11-16 | 成都长城开发科技有限公司 | Microservice rapid integration system, method, device and storage medium |
| CN116155715A (en) * | 2023-02-15 | 2023-05-23 | 北京比特易湃信息技术有限公司 | A Service Governance Platform |
| CN117609973A (en) * | 2024-01-23 | 2024-02-27 | 江苏博云科技股份有限公司 | Registration center nano-tube method, system, device and server |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115442372A (en) * | 2022-09-16 | 2022-12-06 | 平安付科技服务有限公司 | Interface calling method and micro-service application system applying same |
| CN115269612B (en) * | 2022-09-26 | 2022-12-13 | 公安部信息通信中心 | Cross-platform multi-dimensional data fusion system based on micro-service |
| CN115934765B (en) * | 2022-10-11 | 2024-12-06 | 中电金信软件(上海)有限公司 | Data sharing method, device and electronic device between microservice systems |
| CN115604333B (en) * | 2022-10-12 | 2023-09-12 | 江苏赛融科技股份有限公司 | Distributed big data analysis service scheduling method and system based on dubbo |
| CN115914351A (en) * | 2022-11-30 | 2023-04-04 | 中国银行股份有限公司 | Data processing engine system and computer equipment |
| CN116647552A (en) * | 2022-12-12 | 2023-08-25 | 中移信息技术有限公司 | Service processing method and system, terminal and storage medium in heterogeneous microservice cluster |
| CN116346773A (en) * | 2023-03-31 | 2023-06-27 | 河南合众伟奇云智科技有限公司 | Single registration method under multi-registration center environment |
| CN116975477A (en) * | 2023-07-31 | 2023-10-31 | 山东浪潮科学研究院有限公司 | An application service calling method, device, equipment and storage medium |
| CN118714191B (en) * | 2024-08-28 | 2024-11-15 | 西安奇点能源股份有限公司 | Method for unified registration, discovery and calling of multi-type service |
| CN119829363A (en) * | 2024-12-11 | 2025-04-15 | 中国工商银行股份有限公司 | Micro-service operation control method and device, storage medium and electronic equipment |
| CN120233999A (en) * | 2025-05-27 | 2025-07-01 | 深圳渊联技术有限公司 | Microservice decoupling method, device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180032534A1 (en) * | 2016-07-27 | 2018-02-01 | Sap Se | Analytics mediation for microservice architectures |
| CN109547570A (en) * | 2018-12-29 | 2019-03-29 | 深圳云天励飞技术有限公司 | Service registration method, apparatus, registration center's management equipment and storage medium |
| CN111767095A (en) * | 2020-06-30 | 2020-10-13 | 平安国际智慧城市科技股份有限公司 | Microservice generation method, apparatus, terminal device and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109246251B (en) * | 2018-11-13 | 2021-01-22 | 杭州数梦工场科技有限公司 | Micro-service calling method, device, system, equipment and readable storage medium |
| CN109714319A (en) * | 2018-12-06 | 2019-05-03 | 深圳市中农网有限公司 | Management system, method, apparatus, computer equipment and the storage medium of micro services |
| CN110262902B (en) * | 2019-05-16 | 2021-08-31 | 杭州朗和科技有限公司 | Information processing method and system, medium, and computing device |
| CN110392114B (en) * | 2019-07-26 | 2022-02-08 | 中国工商银行股份有限公司 | Service call processing system and method across multiple registration centers |
| CN110381163B (en) * | 2019-07-30 | 2022-09-20 | 普信恒业科技发展（北京）有限公司 | Method for forwarding service request by gateway node and gateway node |
| CN110990047B (en) * | 2019-11-06 | 2021-11-19 | 北京云思畅想科技有限公司 | Fusion method and device for multiple microservice architectures |
-
2020
- 2020-12-24 CN CN202011556562.6A patent/CN112738060B/en active Active
-
2021
- 2021-03-31 WO PCT/CN2021/084251 patent/WO2022134358A1/en not_active Ceased
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180032534A1 (en) * | 2016-07-27 | 2018-02-01 | Sap Se | Analytics mediation for microservice architectures |
| CN109547570A (en) * | 2018-12-29 | 2019-03-29 | 深圳云天励飞技术有限公司 | Service registration method, apparatus, registration center's management equipment and storage medium |
| CN111767095A (en) * | 2020-06-30 | 2020-10-13 | 平安国际智慧城市科技股份有限公司 | Microservice generation method, apparatus, terminal device and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113656096A (en) * | 2021-08-10 | 2021-11-16 | 成都长城开发科技有限公司 | Microservice rapid integration system, method, device and storage medium |
| CN116155715A (en) * | 2023-02-15 | 2023-05-23 | 北京比特易湃信息技术有限公司 | A Service Governance Platform |
| CN117609973A (en) * | 2024-01-23 | 2024-02-27 | 江苏博云科技股份有限公司 | Registration center nano-tube method, system, device and server |
| CN117609973B (en) * | 2024-01-23 | 2024-05-07 | 江苏博云科技股份有限公司 | Registration center nano-tube method, system, device and server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112738060B (en) | 2022-11-18 |
| WO2022134358A1 (en) | 2022-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112738060B (en) | Method and device for processing micro-service data, micro-service processing platform and medium | |
| CN107729139B (en) | Method and device for concurrently acquiring resources | |
| US7673029B2 (en) | Grid automation bus to integrate management frameworks for dynamic grid management | |
| CN102004745B (en) | Data transfer system and method | |
| EP3660665A1 (en) | Business processing method, apparatus, device and system using the same, and readable storage medium of the same | |
| CN113032125B (en) | Job scheduling method, job scheduling device, computer system and computer readable storage medium | |
| CN111026367A (en) | Micro-service arranging method and device, terminal equipment and storage medium | |
| CA2904253C (en) | Computer system using in-service software upgrade | |
| CN112187933B (en) | Method and system for monitoring services in a multi-architecture cloud platform | |
| CN114327484A (en) | K8S integration and deployment method, system and storage medium supporting multi-architecture | |
| CN113191889B (en) | Wind control configuration method, configuration system, electronic equipment and readable storage medium | |
| CN114168297A (en) | Method, device, equipment and medium for scheduling collection tasks | |
| CN113835830A (en) | AI-based RPA cluster management method, device and storage medium | |
| CN117435367A (en) | User behavior processing method, device, equipment, storage medium and program product | |
| CN116302708A (en) | Data backup method, device, equipment and storage medium based on load balancing | |
| CN117931943B (en) | Data synchronization method, device and equipment based on external interface configurable adapter | |
| CN113535439A (en) | Service request processing method, device, device and storage medium | |
| US9323509B2 (en) | Method and system for automated process distribution | |
| CN116820430B (en) | Asynchronous read-write method, device, computer equipment and storage medium | |
| CN118034861A (en) | Cloud computing system, cloud desktop deployment method and cloud desktop deployment device | |
| CN113342540A (en) | Terminal user payment starting method, system, computer equipment and storage medium | |
| CN110489392A (en) | Data access method, device, system, storage medium and equipment between multi-tenant | |
| CN114610413A (en) | Method, device, equipment and storage medium for executing synchronous and asynchronous tasks based on Java | |
| CN115373886A (en) | Service group container shutdown method, device, computer equipment and storage medium | |
| CN113741912A (en) | Model management system, method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |