CN114115913A - Deployment method, device and computer-readable storage medium of a big data platform - Google Patents
Deployment method, device and computer-readable storage medium of a big data platform Download PDFInfo
- Publication number
- CN114115913A CN114115913A CN202111240350.1A CN202111240350A CN114115913A CN 114115913 A CN114115913 A CN 114115913A CN 202111240350 A CN202111240350 A CN 202111240350A CN 114115913 A CN114115913 A CN 114115913A
- Authority
- CN
- China
- Prior art keywords
- component
- resource
- deployment
- current
- components
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/63—Image based installation; Cloning; Build to order
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Stored Programmes (AREA)
Abstract
The application discloses a deployment method and a deployment device of a big data platform and a computer readable storage medium, wherein the deployment method of the big data platform comprises the following steps: selecting a current deployment component from a preset configuration component set by adopting a guide installation module based on a configuration instruction, generating a component list and installing the current deployment component, wherein the component list comprises the current deployment component; acquiring residual resource data of a host machine by adopting a containerization platform; calculating resource data required by each current deployment component by adopting a resource calculation module based on the residual resource data, and generating a resource configuration list, wherein the resource configuration list comprises the resource data required by each current deployment component; and deploying the big data platform by adopting a containerization platform based on the resource configuration list. Through the mode, the resource data of the user-defined component can be automatically configured, and one-click deployment of the big data platform is completed.
Description
Technical Field
The application relates to the technical field of big data, in particular to a deployment method and device of a big data platform and a computer readable storage medium.
Background
At present, the deployment of a big data platform can only carry out resource allocation and deployment on each component independently in a manual mode, and the configuration is troublesome and time-consuming; moreover, the types of components involved in the deployment of a large data platform are very many, for example, a large-capacity storage component, a high-performance data calculation and analysis component, a monitoring component, a security authentication component and the like, which wastes manpower and material resources, has extremely low deployment efficiency, cannot meet customized scenes, cannot face hardware configurations of different servers, and achieves automatic resource allocation.
Disclosure of Invention
The application provides a deployment method and device of a big data platform and a computer readable storage medium, which can automatically configure resource data of a user-defined assembly and complete one-click deployment of the big data platform.
In order to solve the technical problem, the technical scheme adopted by the application is as follows: a deployment method of a big data platform is provided, and comprises the following steps: selecting a current deployment component from a preset configuration component set by adopting a guide installation module based on a configuration instruction, generating a component list and installing the current deployment component, wherein the component list comprises the current deployment component; acquiring residual resource data of a host machine by adopting a containerization platform; calculating resource data required by each current deployment component by adopting a resource calculation module based on the residual resource data, and generating a resource configuration list, wherein the resource configuration list comprises the resource data required by each current deployment component; and deploying the big data platform by adopting a containerization platform based on the resource configuration list.
In order to solve the above technical problem, another technical solution adopted by the present application is: the deployment device of the big data platform comprises a memory and a processor which are connected with each other, wherein the memory is used for storing a computer program, and the computer program is used for realizing the deployment method of the big data platform in the technical scheme when being executed by the processor.
In order to solve the above technical problem, another technical solution adopted by the present application is: the deployment device of the big data platform comprises: the system comprises a guide installation module, a containerization platform and a resource calculation module; the guide installation module is used for selecting a current deployment component from a preset configuration component set based on the configuration instruction, generating a component list and installing the current deployment component, wherein the component list comprises the current deployment component; the containerization platform is used for acquiring residual resource data of the host machine; the resource calculation module is connected with the guide installation module and the containerization platform and used for calculating resource data required by each current deployment component based on the residual resource data and generating a resource configuration list, and the resource configuration list comprises the resource data required by each current deployment component; the containerization platform is further used for deploying the big data platform based on the resource configuration list.
In order to solve the above technical problem, another technical solution adopted by the present application is: a computer-readable storage medium is provided, which is used for storing a computer program, and when the computer program is executed by a processor, the computer program is used for implementing the deployment method of the big data platform in the above technical solution.
Through the scheme, the beneficial effects of the application are that: the method comprises the steps that a guide installation module, a containerization platform and a resource calculation module are included, the guide installation module selects current deployment components from a preset configuration component set according to a configuration instruction input by a user to generate a component list, and all current deployment components in the component list are installed; then adopting a containerization platform to obtain the residual resource data of the host machine; calculating resource data required by each current deployment component according to the residual resource data through a resource calculation module to generate a resource configuration list, so that the containerized platform deploys the big data platform according to the resource configuration list; by arranging the guide deployment module, the operation of a user can be facilitated to generate a configuration instruction, the current required component can be selected in a user-defined mode according to the actual application requirement of the user, the application requirements of different users can be met, and the method can be applied to a customized scene; and the resource calculation module can automatically allocate corresponding resource data to each current deployment component according to the hardware resource data of the host machine, so that the automatic configuration of component resources is realized, the efficiency of component resource deployment can be improved, a user only needs to select a desired component, the hardware resources of each component do not need to be manually set by the user, one-key deployment is realized, the operation of the user is greatly simplified, and the use by the user is facilitated.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:
FIG. 1 is a schematic structural diagram of an embodiment of a deployment apparatus for a big data platform provided in the present application;
FIG. 2 is a schematic flowchart of an embodiment of a deployment method for a big data platform provided in the present application;
FIG. 3 is a schematic view of a guidance interface provided herein;
FIG. 4 is a schematic flowchart of another embodiment of a deployment method for a big data platform provided in the present application;
FIG. 5 is a schematic structural diagram of an embodiment of a resource calculation module provided herein;
FIG. 6 is a schematic flow chart diagram illustrating one embodiment of step 44 provided herein;
FIG. 7 is a schematic structural diagram of an embodiment of a deployment apparatus for a big data platform provided in the present application;
FIG. 8 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be noted that the following examples are only illustrative of the present application, and do not limit the scope of the present application. Likewise, the following examples are only some examples and not all examples of the present application, and all other examples obtained by a person of ordinary skill in the art without any inventive work are within the scope of the present application.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
It should be noted that the terms "first", "second" and "third" in the present application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of the feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless explicitly specifically limited otherwise. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an embodiment of a deployment apparatus for a big data platform provided in the present application, where a deployment apparatus 10 for a big data platform includes a boot installation module 11, a containerization platform 12, and a resource calculation module 13, and the deployment apparatus 10 for a big data platform is used to implement a deployment method for a big data platform provided in the present application, so as to deploy the big data platform.
The guidance installation module 11 is configured to select a current deployment component from a preset configuration component set based on the configuration instruction, generate a component list, and install the current deployment component; the containerization platform 12 is used for acquiring the residual resource data of the host machine 20; the resource calculation module 13 is connected to the boot installation module 11 and the containerization platform 12, and is configured to calculate resource data required by each currently deployed component based on the remaining resource data, and generate a resource configuration list, where the resource configuration list includes the resource data required by each currently deployed component; the containerization platform 12 is further configured to deploy the big data platform based on the resource configuration list.
Specifically, the following embodiments describe in detail a deployment method of a big data platform adopted by the deployment apparatus of the big data platform.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating an embodiment of a deployment method of a big data platform provided in the present application, where the method includes:
step 21: and selecting a current deployment component from a preset configuration component set by adopting a guide installation module based on the configuration instruction, generating a component list and installing the current deployment component.
The component list comprises current deployment components, a configuration instruction input by a user can be received by adopting the guide installation module, then the deployment components corresponding to the configuration instruction are selected from the preset configuration component set, the component list is generated according to the selected current deployment components, and the current deployment components are installed to prepare for subsequent component resource configuration.
Specifically, the preset configuration component set may include a mandatory component and an optional component, where the mandatory component and the optional component include a basic class component, a security authentication class component, a computation scheduling class component, a monitoring class component, or a storage class component; the mandatory components are necessary components to be installed when the big data platform is deployed, such as: the system comprises a basic component, a calculation scheduling component and a storage component, wherein optional components are unnecessary components selected by a user according to actual application requirements, such as: a security authentication type component or a monitoring type component, etc.
It is understood that each type of component may include one or more similar components, as shown in fig. 3, a guidance interface for user selection is displayed on a display device, the storage class component may include a mass data software platform component (Hadoop), a mass-Parallel Processing architecture component (Massive-Parallel Processing, Mpp), and an architecture framework component (elastic search, ES) for user selection, and the computation scheduling class component may include a Hadoop-based data warehouse component (Hive), a cluster resource management system component (yan), a distributed computation framework component (Mapreduce), an open-source data analysis cluster computation framework component (Spark), and a Master-Slave style-based architecture component (flip) for user selection; it is to be understood that, in order to avoid the situation that the user does not select the mandatory component, which results in the failure of the deployment of the large data platform, the mandatory component may be directly selected by default, and the user is not provided with a reducible selection operation, for example: a distributed collaboration service (zookeeper) component in the base class component, and a configuration hub component.
The configuration instruction comprises a first selection instruction and a second selection instruction, wherein the first selection instruction is used for selecting the necessary components, and the second selection instruction is used for selecting the necessary components; the user can select the corresponding component type according to the requirements of the actual application scene, for example: when a user needs to set a security mode, the user can set the security mode by selecting a security authentication class component, such as network authentication protocol components (Kerberors) and a centralized security management framework component (range); and the security authentication class component may not be selected when the security mode does not need to be set.
Step 22: and acquiring the residual resource data of the host machine by adopting a containerization platform.
The components of the containerization platform may include an open-source application container engine component (Docker) and an agent component (Kubernet), and the containerization platform is built on a host (i.e., a server), and the containerization platform is used to monitor hardware environment resources in the host, so as to obtain remaining resource data of the host, specifically, the remaining resource data includes the number of Solid State Drives (SSD), the size of the SSD), the number of Hard Disk drives (Hard-Disk drives, HDDs), the size of the Hard Disk drives, the number of cores of a Central Processing Unit (CPU), the remaining space of a memory, or an affinity policy, where the affinity policy is whether the component can be configured on the same host with other components.
Furthermore, the containerization platform can be built in a host machine cluster comprising a plurality of host machines, the affinity can be used for representing the main-standby relation among the components, the components with the main-standby relation can be configured on different host machines under the host machine cluster through the identification affinity strategy, so that the stability of the host machine cluster is ensured, other host machines can be normally used under the condition that one host machine is abnormal, and the condition that all the host machines in the whole cluster cannot be used (namely are hung up) at the same time is prevented.
Step 23: and calculating the resource data required by each current deployment component by adopting a resource calculation module based on the residual resource data, and generating a resource configuration list.
The resource configuration list comprises resource data required by each current deployment component, the containerization platform can be used for sending the residual resource data to the resource calculation module, and the guide installation module is used for sending the component list to the resource calculation module, so that the resource calculation module performs resource configuration on the current deployment component according to the residual resource data in the host machine, and the residual resource data are reasonably distributed to the current deployment component, for example: the residual memory in the host machine is 20G, and the current deployment components A-C contained in the component list can reasonably distribute the residual 20G memory to the current deployment components A-C at the moment so as to meet the requirement of each current deployment component; it can be understood that the more resource data allocated to a component, the better the working performance of the component, and in general, in the case that the remaining resource data is sufficient, all the remaining resource data can be reasonably allocated to all the components on the basis of meeting the minimum resource requirement of all the components in the component list, for example: the memory resources of 20G can be all allocated to the current deployment components A-C according to a preset proportion so as to generate a corresponding resource configuration list.
Step 24: and deploying the big data platform by adopting a containerization platform based on the resource configuration list.
The containerization platform performs resource configuration on the container mirror image of the component according to the resource data required by each currently deployed component in the resource configuration list, where the container mirror image of the component may include: the deployment installation package, the resource configuration of the components and the execution statement are completed in the container mirror image of the components according to the resource configuration list, and the corresponding component container can be established, so that the deployment installation package is operated in the component container, and the deployment of the large data platform is realized.
In a specific embodiment, the resource configuration manifest may further include an affinity allocation scheme (i.e., an affinity policy), and the containerization platform may be adopted to allocate the components in different hosts according to the affinity allocation scheme, so as to ensure the stability of the host cluster. The containerization platform can also be used for scheduling containers and monitoring the states of the containers, when the containers are hung dead due to abnormal conditions, the containers can be pulled up, namely, the functions of the containers are recovered, and when the containers are pulled up for multiple times and are invalid, a new container with the same function can be called, so that the new container runs and deploys the installation package, the usability of the established containers can be improved, and the stability of deployment of the large data platform is further improved.
In the embodiment, by setting the guide deployment module, a user can self-define and select the current deployment component according to the application requirement, so that the application requirements of different users can be met, and the method can be applied to a customized scene; meanwhile, by arranging the resource calculation module, the resource data can be reasonably distributed for each current deployment component automatically without manual resource distribution, the automatic configuration of component resources is realized, the deployment efficiency of a big data platform is improved, and the components with affinity relation are distributed to different host machines while the resource data are distributed by considering an affinity distribution scheme, so that the stability of a host machine cluster can be improved; moreover, a user only needs to select a desired component on the display interface, one-click deployment is realized, hardware resources of each component do not need to be manually set by the user, operation of the user is greatly simplified, and the user can use the system conveniently.
Referring to fig. 4, fig. 4 is a schematic flowchart illustrating a deployment method of a big data platform according to another embodiment of the present application, where the method includes:
step 41: and selecting a current deployment component from a preset configuration component set by adopting a guide installation module based on the configuration instruction, generating a component list and installing the current deployment component.
Step 42: and acquiring the residual resource data of the host machine by adopting a containerization platform.
In a specific embodiment, as shown in fig. 5, the resource calculation module 13 may include a resource collection module 131 and a calculation module 132, and after acquiring the component list and acquiring the remaining resource data of the host, the resource calculation module is adopted to calculate resource data required by each currently deployed component according to the remaining resource data, so as to generate a resource configuration list, specifically including the following steps 43 to 44:
step 43: and acquiring residual resource data from the containerization platform by adopting a resource collection module.
The resource collection module is connected with the containerization platform, and can acquire residual resource data from the containerization platform by adopting the resource collection module, wherein the residual resource data comprises the number of the solid state disks, the size of the solid state disks, the number of the hard disk drives, the size of the hard disk drives, the number of cores of the central processing unit, and the residual space or affinity strategy of the memory.
Step 44: and performing resource allocation processing by adopting a computing module based on the residual resource data to obtain resource data required by all the current deployment components.
As shown in fig. 5, the calculation module 132 may further include a first calculation module 1321 and a second calculation module 1322 connected to each other, and the first calculation module 1321 and the second calculation module 1322 may be adopted to perform resource allocation processing based on the remaining resource data to obtain resource data required by all currently deployed components, where the allocation method is shown in steps 441 to 445 in fig. 6.
Step 441: and matching the component list with the component resource allocation table by adopting a second calculation module to obtain the current resource allocation ratio.
The first calculation module is used for storing the component resource configuration table, and the second calculation module is used for matching the component list with the component resource configuration table to obtain a current resource configuration ratio corresponding to the component list in the component resource configuration table, wherein the current resource configuration ratio is the ratio of resource data required by all current deployed components.
Specifically, the component resource allocation table may include a resource allocation ratio of any at least two preset components, that is, a resource allocation ratio corresponding to a possible permutation and combination among all components included in the preset configuration component set is preset, for example: the preset configuration component set comprises components A-C, wherein the component A is a required component, the component B, C is an optional component, three resource configuration ratios can be preset in a component resource configuration table, namely the resource configuration ratio of the component A to the component B, the resource configuration ratio of the component A to the component C and the resource configuration ratios of the components A-C, and then a second calculation module is adopted to match the combination of the current deployment components contained in the component list with the components in the component resource configuration table; for example: the component list includes components a to C, and the resource allocation ratio corresponding to the combination of the components a to C can be found in the component resource allocation table, as shown in fig. 2: 3: 5, then the ratio of 2: 3: and 5, carrying out resource allocation on the components A to C by the resource allocation comparison.
Step 442: and processing the current resource allocation ratio and the residual resource data by adopting a second calculation module to obtain the resource data required by the current deployment assembly.
After the current resource allocation ratio of the current deployment assembly is obtained through matching, the ratio of the resource data required by the current deployment assembly can be multiplied by the residual resource data to obtain the resource data required by the current deployment assembly; for example: the resource allocation ratio of the current deployment components A-C is 2: 3: 5, if the remaining memory of the host is 10G, then the ratio of 2: 3: multiplying the residual resource data by 10G to obtain resource data required by the current deployment assemblies A-C, and respectively allocating memory resources of 2G, 3G and 5G to the assemblies A-C; if the current deployment components are only component a and component B, then the method can be as follows 2: a ratio of 3 allocates resources to component a and component B.
Step 443: and judging whether the resource data required by the current deployment assembly falls in the resource range corresponding to the current deployment assembly by adopting a second calculation module.
The component resource configuration table further includes a resource range of each component, that is, a lowest resource lower limit value and an optimal resource upper limit value of each component are included, and when resource data is allocated to the currently deployed component, it is further considered that the resource data allocated to each currently deployed component is greater than or equal to the respective corresponding lowest resource lower limit value, so as to prevent the resource data allocated to the currently deployed component from being insufficient and causing the currently deployed component to fail to operate normally, at this time, a second computing module may be used to determine whether the resource data required by the currently deployed component falls within the resource range corresponding to the currently deployed component, that is, whether the resource data is greater than or equal to the lowest resource lower limit value, so as to determine whether the resource allocated to each currently deployed component is sufficient.
It can be understood that the more resources allocated to a component, the better the performance that the component can achieve, and generally, the optimal upper limit value of the resource in the resource range of the component has no practical limitation, that is, the component does not actually set a fixed optimal upper limit value of the resource, and the optimal upper limit value of the resource that the component can have is that all the resource data currently remaining in the host, that is, all the remaining resource data is configured to one component.
Step 444: and if the resource data required by the current deployment assembly does not fall within the resource range corresponding to the current deployment assembly, generating a reminding message, and modifying the current resource configuration ratio so as to enable the modified resource data required by the current deployment assembly to fall within the resource range corresponding to the current deployment assembly.
When the resource data required by the current deployment assembly does not fall in the resource range corresponding to the current deployment assembly, it is indicated that the resource allocated to the current deployment assembly is insufficient, and the minimum resource lower limit value corresponding to the current deployment assembly is not met, a reminding message similar to 'resource configuration insufficiency' is generated and displayed on a display device, and meanwhile, the second computing module is adopted to adjust the current resource allocation to the current deployment assembly, so that the resource data required by the current deployment assembly falls in the resource range corresponding to the current deployment assembly.
In a specific embodiment, the second calculation module may be used to modify the current resource allocation ratio, for example: taking the currently deployed components as components a to C as an example, the lowest resource lower limit value corresponding to the component a is 4G, the lowest resource lower limit value corresponding to the component B is 4G, the lowest resource lower limit value corresponding to the component C is 2G, and the resource allocation ratio value corresponding to the components a to C included in the component resource allocation table is 4: 3: 3, if the remaining memory of the host is 10G, then the memory resources allocated to the components a to C according to the resource allocation ratio are respectively 4G, 3G, and 3G, it can be known that the resource data 3G allocated to the component B does not reach the lowest resource lower limit value 4G corresponding to the component B, and thus the component B may not normally operate, and at this time, the resource allocation ratio may be adjusted by using the second calculation module, according to the new resource allocation ratio 4: 4: 2, allocating resource data, wherein the memory resources reallocated to the components A-C are respectively 4G, 4G and 2G, so that the lowest resource lower limit value of the component B is met, and the component B can normally operate; specifically, after the resource allocation ratio is adjusted, whether to update the adjusted resource allocation ratio to the component resource allocation table may be selected according to actual application requirements.
It can be understood that, in other embodiments, when the remaining resource data of the host is insufficient and cannot meet the minimum resource lower limit value of all currently deployed components, that is, when the sum of the remaining resource data of the host is smaller than the sum of the minimum resource lower limit values of all currently deployed components, a warning message similar to "insufficient remaining resources" may also be generated and displayed on the display device to remind the user to reselect the currently deployed component, generate a new component list, and then perform resource allocation again based on the new component list.
Step 445: and if the resource data required by the current deployment assembly falls into the resource range corresponding to the current deployment assembly, updating the application template by using the resource data required by the current deployment assembly, and running a deployment installation package to deploy the big data platform.
When the resource data required by the current deployment component falls within the resource range corresponding to the current deployment component, that is, the lowest resource lower limit value of all the current deployment components is met, all the current deployment components can normally operate, at this time, the deployment of the big data platform can be completed according to the resource data required by the current deployment component, the application template can be updated by using the resource data required by the current deployment component, and the deployment installation package is operated to complete the deployment of the big data platform.
Further, the second computing module is also used for performing affinity configuration on the currently deployed component according to the affinity characteristics of the component, for example: the components A and C are deployed at present, wherein the components A and the components C have affinity relations, and the components A and the components C are configured in different host machines while the application template is updated by using resource data required by the current deployed components, so that the stability of the host machine cluster is ensured.
In a specific embodiment, after completing the configuration of the resource data of the current deployment component, the user may further select to adjust the current deployment component, that is, delete/add the current deployment component, and generate a new component list; specifically, when a user adds a new currently deployed component, a new current resource configuration ratio may be re-matched based on the component resource configuration table to perform resource allocation on the adjusted currently deployed component.
Further, the component resource allocation table further comprises a weight value of each component, wherein the weight value can be 1-100 or 0-1, and a specific numerical value range can be set according to application requirements; when the current deployment component is deleted, the resource distribution can be carried out again based on the weight value of the current deployment component, and when the deleted current deployment component exists in the component list, the resource data released by the deleted current deployment component is distributed to the undeleted current deployment component with the highest weight value; for example: the current resource configuration is that memory resources of 2G, 3G and 5G are respectively allocated to current deployment assemblies A-C, then the current deployment assembly B is deleted, the memory resource of 3G is released, the weight value of the assembly A is 0.6, the weight value of the assembly C is 0.8, and at the moment, the released memory resource of 3G can be directly allocated to the assembly C with a higher weight value.
Or distributing the resources corresponding to the deleted current deployment component to all the undeleted current deployment components according to the ratio of the weight values of all the undeleted current deployment components; for example: the ratio of the weight values of the component A to the component C is 3:4, and the released 3G memory resources can be distributed to the component A and the component C according to the ratio of 3: 4.
In the technical scheme provided by this embodiment, the second computing module is adopted to find the resource configuration ratio of the currently deployed component based on the preset component resource configuration table, so as to allocate the remaining resource data of the host according to the resource configuration ratio, and realize automatic reasonable allocation of the resource of the currently deployed component; and when the resource data configuration of the current deployment assembly is insufficient, the current resource configuration ratio can be automatically adjusted to ensure that each current deployment assembly can normally operate, and released resource data can be distributed to the remaining current deployment assemblies according to weight values when the current deployment assembly is deleted, so that the utilization rate of the resource data can be increased, the operation performance of the assemblies can be optimized, and the efficiency and the stability of the deployment of the large data platform can be further improved.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a deployment apparatus for a big data platform provided in the present application, where the deployment apparatus 70 for a big data platform includes a memory 71 and a processor 72 that are connected to each other, the memory 71 is used for storing a computer program, and the computer program, when executed by the processor 72, is used for implementing a deployment method for a big data platform in the foregoing embodiment.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an embodiment of a computer-readable storage medium 80 provided in the present application, where the computer-readable storage medium 80 is used to store a computer program 81, and when the computer program 81 is executed by a processor, the computer program is used to implement the deployment method of the big data platform in the foregoing embodiment.
The computer readable storage medium 80 may be a server, a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules or units is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111240350.1A CN114115913A (en) | 2021-10-25 | 2021-10-25 | Deployment method, device and computer-readable storage medium of a big data platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111240350.1A CN114115913A (en) | 2021-10-25 | 2021-10-25 | Deployment method, device and computer-readable storage medium of a big data platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114115913A true CN114115913A (en) | 2022-03-01 |
Family
ID=80377313
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111240350.1A Pending CN114115913A (en) | 2021-10-25 | 2021-10-25 | Deployment method, device and computer-readable storage medium of a big data platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114115913A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116301934A (en) * | 2023-02-17 | 2023-06-23 | 深圳市金蝶天燕云计算股份有限公司 | Software installation method, device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9329907B1 (en) * | 2014-12-18 | 2016-05-03 | International Business Machines Corporation | Automated exploitation of virtual machine resource modifications |
| CN109117148A (en) * | 2018-07-12 | 2019-01-01 | 湖北省楚天云有限公司 | A kind of method and system of the application deployment on cloud computing platform |
| CN111580958A (en) * | 2020-04-20 | 2020-08-25 | 佛山科学技术学院 | A method and device for deploying a big data platform |
| CN111857736A (en) * | 2020-07-28 | 2020-10-30 | 中国建设银行股份有限公司 | Cloud computing product generation method, device, equipment and storage medium |
-
2021
- 2021-10-25 CN CN202111240350.1A patent/CN114115913A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9329907B1 (en) * | 2014-12-18 | 2016-05-03 | International Business Machines Corporation | Automated exploitation of virtual machine resource modifications |
| CN109117148A (en) * | 2018-07-12 | 2019-01-01 | 湖北省楚天云有限公司 | A kind of method and system of the application deployment on cloud computing platform |
| CN111580958A (en) * | 2020-04-20 | 2020-08-25 | 佛山科学技术学院 | A method and device for deploying a big data platform |
| CN111857736A (en) * | 2020-07-28 | 2020-10-30 | 中国建设银行股份有限公司 | Cloud computing product generation method, device, equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116301934A (en) * | 2023-02-17 | 2023-06-23 | 深圳市金蝶天燕云计算股份有限公司 | Software installation method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11966768B2 (en) | Apparatus and method for multi-cloud service platform | |
| CN111385114B (en) | VNF service instantiation method and device | |
| US12333327B2 (en) | Coordinated container scheduling for improved resource allocation in virtual computing environment | |
| US10635496B2 (en) | Thread pool management | |
| EP3637733B1 (en) | Load balancing engine, client, distributed computing system, and load balancing method | |
| CN105049268B (en) | distributed computing resource allocation system and task processing method | |
| JP6514241B2 (en) | Service orchestration method and apparatus in software defined network, storage medium | |
| CN113296792B (en) | Storage method, device, equipment, storage medium and system | |
| CN108076156B (en) | A hybrid cloud system based on Chinese cloud products | |
| US20080263553A1 (en) | Dynamic Service Level Manager for Image Pools | |
| CN110221920B (en) | Deployment method, device, storage medium and system | |
| CN110351384A (en) | Big data platform method for managing resource, device, equipment and readable storage medium storing program for executing | |
| WO2022002148A1 (en) | Resource scheduling method, resource scheduling system, and device | |
| CN103365713A (en) | Resource dispatch and management method and device | |
| CN112637304B (en) | A cross-cloud resource processing system and resource management method | |
| CN114090176A (en) | Kubernetes-based container scheduling method | |
| CN112437129B (en) | Cluster management method and cluster management device | |
| WO2020063550A1 (en) | Policy decision method, apparatus and system, and storage medium, policy decision unit and cluster | |
| CN110838939B (en) | Scheduling method based on lightweight container and edge Internet of things management platform | |
| CN114564314A (en) | Method and device for collecting monitoring data in cluster | |
| CN118051341A (en) | Computing power resource scheduling method, computing power resource scheduling device, terminal equipment and storage medium | |
| CN113297031A (en) | Container group protection method and device in container cluster | |
| CN114115913A (en) | Deployment method, device and computer-readable storage medium of a big data platform | |
| CN114745377A (en) | Edge cloud cluster service system and implementation method | |
| CN114579298A (en) | Resource management method, resource manager, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |