CN119094335B - A method for automatically configuring data center resources - Google Patents
A method for automatically configuring data center resources Download PDFInfo
- Publication number
- CN119094335B CN119094335B CN202411570176.0A CN202411570176A CN119094335B CN 119094335 B CN119094335 B CN 119094335B CN 202411570176 A CN202411570176 A CN 202411570176A CN 119094335 B CN119094335 B CN 119094335B
- Authority
- CN
- China
- Prior art keywords
- data
- resource
- business
- service
- demand
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000013468 resource allocation Methods 0.000 claims abstract description 58
- 238000004458 analytical method Methods 0.000 claims abstract description 52
- 238000003860 storage Methods 0.000 claims abstract description 33
- 238000012544 monitoring process Methods 0.000 claims abstract description 28
- 238000005457 optimization Methods 0.000 claims description 50
- 238000012549 training Methods 0.000 claims description 35
- 238000011156 evaluation Methods 0.000 claims description 34
- 238000007726 management method Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 17
- 238000012795 verification Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 238000010200 validation analysis Methods 0.000 claims description 10
- 238000004140 cleaning Methods 0.000 claims description 9
- 230000003247 decreasing effect Effects 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000008901 benefit Effects 0.000 description 13
- 230000000694 effects Effects 0.000 description 10
- 238000013439 planning Methods 0.000 description 10
- 230000001737 promoting effect Effects 0.000 description 10
- 230000008859 change Effects 0.000 description 8
- 239000002699 waste material Substances 0.000 description 8
- 238000011160 research Methods 0.000 description 7
- 239000013598 vector Substances 0.000 description 6
- 238000011161 development Methods 0.000 description 5
- 230000006872 improvement Effects 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000014759 maintenance of location Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 239000000047 product Substances 0.000 description 3
- 238000012163 sequencing technique Methods 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 238000012300 Sequence Analysis Methods 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000012854 evaluation process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000001932 seasonal effect Effects 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000011068 loading method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- YHXISWVBGDMDLQ-UHFFFAOYSA-N moclobemide Chemical compound C1=CC(Cl)=CC=C1C(=O)NCCN1CCOCC1 YHXISWVBGDMDLQ-UHFFFAOYSA-N 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0876—Aspects of the degree of configuration automation
- H04L41/0886—Fully automatic configuration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
- H04L41/082—Configuration setting characterised by the conditions triggering a change of settings the condition being updates or upgrades of network functionality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Automation & Control Theory (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a method for automatically configuring data center resources, which relates to the technical field of dynamic resource allocation, and comprises the steps of monitoring load data of a data center in real time, and obtaining real-time use states of all resources, wherein the real-time use states comprise processor use rate, memory occupancy rate, network bandwidth use rate and use condition of storage equipment; and collecting and analyzing the current running service and the service demand data to be online in real time according to the load data of the data center, wherein the service demand data comprises service types, states, resource consumption, expected online time and expected resource demands, so as to obtain service analysis results. According to the invention, through real-time monitoring, intelligent analysis and prediction, dynamic optimal configuration of the data center resources is realized, the resource utilization rate and service continuity are effectively improved, and meanwhile, the operation cost is reduced.
Description
Technical Field
The invention relates to the technical field of dynamic resource allocation, in particular to a method for automatically configuring data center resources.
Background
Conventional data centers are designed and built to take into account the maximum amount of load and to leave room for upgrade improvement for the future, but this approach may limit the flexibility and scalability of the data center. When new services need to be deployed quickly or resource allocation needs to be adjusted, some traditional methods respond slowly and cannot meet the service requirements. Some traditional data centers may cause high operation cost due to low energy utilization efficiency and high management complexity.
In addition, in the aspect of safety, because some of the safety aspects depend on manual operation and management, data leakage or system faults possibly caused by human errors or malicious behaviors exist, and effective automation tools and processes are lacked to ensure consistency and safety of configuration, so that potential safety hazards are increased.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a method for automatically configuring the data center resources, which improves the utilization rate, flexibility and response speed of the resources and reduces the management complexity and cost.
In order to solve the technical problems, the technical scheme of the invention is as follows:
a method of automatic configuration of data center resources, the method comprising:
the method comprises the steps of monitoring load data of a data center in real time, and obtaining real-time use states of various resources, wherein the real-time use states comprise processor use rate, memory occupancy rate, network bandwidth use rate and use condition of storage equipment;
According to load data of a data center, current operation service and service demand data about to be online are collected and analyzed in real time, wherein the service demand data comprises service types, states, resource consumption, expected online time and expected resource demands, so that service analysis results are obtained;
according to the service analysis result, inputting the load data and the service demand data which are monitored in real time into a resource allocation prediction and optimization model for training and learning, and predicting the resource demand in a future period of time to obtain a final prediction result;
Dynamically allocating and optimizing resources of a data center according to a final prediction result, wherein the dynamic allocation comprises the steps of adjusting the number of processor cores, memory allocation and network bandwidth allocation;
according to the dynamic allocation and optimization of the resources of the data center, the adjustment of the corresponding resource allocation is automatically executed, and the running state of the data center is continuously monitored so as to acquire new load data and service demand data in real time.
Further, load data of the data center, collecting and analyzing current running service and service demand data about to be online in real time, including service type, status, resource consumption, expected online time and expected resource demand, to obtain service analysis results, including:
Integrating the real-time service data and the planning data of the to-be-online service into a unified data platform to obtain an integrated data set;
Cleaning and preprocessing the integrated data set, and extracting key features to obtain a processed data set;
According to the processed data set, analyzing the resource use mode of the current running service, identifying the service type with large resource consumption or rapid growth, and marking the service type as a key service;
according to the marked key service and service rules, expected benefits and customer influence factors, carrying out priority evaluation on the service to be online, and outputting a service priority list;
According to the service priority list and the resource use mode, analyzing the influence of the services with different priorities on the resources to generate a service resource use analysis report;
and according to the service resource usage analysis report, obtaining a service analysis result.
Further, according to the marked key service and service rule, expected benefits and customer influencing factors, the priority evaluation is carried out on the service to be online, and a service priority list is output, which comprises the following steps:
setting evaluation standards and measurement indexes, including the urgency of the service;
For each service to be online, collecting relevant information, including service demand documents and customer feedback, and taking the relevant information as input data of evaluation standards;
Scoring each service to be online according to the set evaluation standard and the key service information;
in the scoring process, corresponding scores are given according to the matching degree of the business and each evaluation standard;
Summarizing the scores of each service to obtain a comprehensive score, sequencing the services according to the comprehensive score, and generating a service priority list from high to low.
Further, according to the service analysis result, load data and service demand data monitored in real time are input into a resource allocation prediction and optimization model to be trained and learned, and resource demands in a period of time in the future are predicted to obtain a final prediction result, which comprises the following steps:
According to the service analysis result, determining the characteristics related to resource demand prediction, including historical load data, service growth trend and service type;
Constructing a resource allocation prediction and optimization model, and determining input items of the optimization model, including load data, business demand data and output items monitored in real time, including resource demand predicted values in a future period of time;
Dividing historical load data and business demand data into a training set, a verification set and a test set, training an optimization model by using the training set data, and evaluating the prediction performance of the optimization model on the verification set by using root mean square error in the training process to obtain an evaluation result;
according to the evaluation result, carrying out iterative optimization on the optimization model, including adjusting parameters;
training, verifying and evaluating are repeated until the performance of the optimized model reaches a preset standard, so as to obtain a final optimized model;
And according to a final optimization model, load data and business demand data which are monitored in real time at present are used as input items, and resource demand predicted values in a period of time in the future are calculated to obtain a final predicted result, wherein the final predicted result comprises the resource demand predicted values, the predicted use amount and the growth trend of various types of resources.
Further, the root mean square error is calculated as:
R;
wherein R represents root mean square error; a total number of samples representing the validation set; Represent the first Actual values of the individual samples; Represent the first Predicted values for the individual samples; Represent the first Weights of the samples; Representing regularization coefficients; Representing the number of model parameters; Represent the first And model parameters.
Further, the calculation formula of the predicted value of the resource demand in a future period of time is as follows:
;
Wherein, Indicating future time of dayResource demand forecast values of (2); An intercept term representing the model; An index representing the influencing factors; Represent the first Weighting of individual influencing factorsRepresent the firstThe time of each influencing factorIs of the observed value of (2)Representing the deviation between the predicted value and the actual value.
Further, according to dynamically allocating and optimizing resources of the data center, automatically executing adjustment of corresponding resource allocation, and continuously monitoring an operation state of the data center to obtain new load data and service demand data in real time, including:
Receiving a final prediction result and automatically triggering a resource allocation adjustment flow;
Determining a corresponding resource allocation adjustment scheme according to the content of the prediction result, wherein the resource allocation adjustment scheme comprises the steps of increasing or decreasing server resources, adjusting network bandwidth allocation or reallocating storage resources;
transmitting the resource allocation adjustment scheme to a management system of the data center, and executing resource allocation adjustment operation;
The operation state of the data center is continuously monitored, and the operation state comprises key indexes of the processor utilization rate, the memory occupancy rate, the network bandwidth utilization rate and the service condition of the storage device, so that new load data and service demand data, including real-time traffic, user request quantity and transaction processing quantity, are obtained in real time.
The scheme of the invention at least comprises the following beneficial effects:
The system can rapidly respond to the change of service demands by monitoring the real-time use state of resources in real time through a data center management system, can rapidly capture the change of the use of the resources, collect and analyze the service demand data of running service and on-line service in real time, and can predict the resource demands in a future period by training and learning through a resource allocation prediction and optimization model, thereby realizing the advanced planning and optimization allocation of the resources. The prediction capability is helpful to avoid the situation of excessive or insufficient resources, improves the utilization rate of the resources and reduces the waste.
And dynamically allocating and adjusting resources of the data center according to the final prediction result, including the number of processor cores, memory allocation, network bandwidth allocation and the like, so that the flexibility and the expandability of the data center are enhanced. The dynamic adjustment capability enables the data center to better adapt to the rapid change of service requirements, and improves the operation efficiency and service quality of the data center. The system automatically executes the adjustment of the corresponding resource configuration according to the dynamic allocation and the optimization result, reduces the requirement of manual intervention, improves the configuration efficiency, reduces the possibility of human errors and improves the stability and the reliability of the data center.
Drawings
Fig. 1 is a flow chart of a method for automatically configuring resources of a data center according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As shown in fig. 1, an embodiment of the present invention proposes a method for automatically configuring resources of a data center, the method including the following steps:
step 11, load data of a data center are monitored in real time, and real-time use states of all resources are obtained, wherein the real-time use states comprise processor use rate, memory occupancy rate, network bandwidth use rate and use condition of storage equipment;
Step 12, according to the load data of the data center, collecting and analyzing the current operation service and the service demand data to be online in real time, wherein the service demand data comprises service type, state, resource consumption, expected online time and expected resource demand, so as to obtain a service analysis result;
Step 13, according to the service analysis result, inputting the load data and the service demand data which are monitored in real time into a resource allocation prediction and optimization model for training and learning, and predicting the resource demand in a period of time in the future to obtain a final prediction result;
Step 14, dynamically allocating and optimizing the resources of the data center according to the final prediction result, wherein the steps include adjusting the core number of the processor, the memory allocation and the network bandwidth allocation;
And step 15, according to the dynamic allocation and optimization of the resources of the data center, automatically executing the adjustment of the corresponding resource allocation, and continuously monitoring the running state of the data center to acquire new load data and service demand data in real time.
In the embodiment of the invention, the administrator can obtain instant feedback about the performance of the data center by monitoring the utilization state of each resource of the data center, such as the utilization rate of a processor, the occupancy rate of a memory and the like in real time. This transparency helps to quickly identify and resolve potential performance bottlenecks or resource misuse issues. The continuous resource status monitoring can detect abnormal conditions, thereby taking precautions before problems occur, reducing unexpected downtime, and improving the availability of the data center. Current and future business demand data are collected and analyzed in real time, so that a data center can know the development trend and the resource demand of business more accurately, and a more timely response is made. By carrying out demand analysis on the service to be online, the data center can carry out prospective resource planning, ensure that enough resource support can be obtained when a new service is online, and avoid service delay or interruption caused by insufficient resources.
By utilizing the resource allocation prediction and optimization model, the data center can predict future resource demands based on historical and real-time data, and scientific basis is provided for resource allocation decision. Accurate resource demand predictions help data centers to balance avoiding resource overages and shortages, thereby reducing operational risk and costs. And the resources are dynamically adjusted according to the final prediction result, so that the data center resources can be ensured to be always kept in a utilization state, and the overall operation efficiency is improved. Dynamic resource allocation enables the data center to more flexibly cope with changes in traffic demand, and can quickly make adjustments whether traffic is suddenly increased or suddenly decreased. The system automatically executes resource allocation adjustment according to the resource allocation result, so that the need of manual intervention is reduced, and the risk caused by manual misoperation is reduced. By continuously monitoring the operating state of the data center and acquiring new data in real time, the system can continuously optimize the resource allocation, and ensure that the data center is always kept in the optimal performance state.
In a preferred embodiment of the present invention, the step 11 of monitoring load data of the data center in real time to obtain real-time usage status of each resource, including usage of a processor, occupancy of a memory, usage of a network bandwidth, and usage of a storage device may include:
Sensors, such as temperature sensors, humidity sensors, voltage and current sensors, etc., are deployed on critical equipment of a data center for collecting operational status data of the equipment in real time. The data collector is responsible for receiving raw data from the sensor and performing preliminary processing and formatting. For example, DHT11, DHT22, etc. are data collectors. The data collector sends the processed data to a monitoring platform of the data center management system through a communication module (such as Ethernet). The communication module ensures real-time and accurate transmission of data. And a standardized data transmission protocol, such as MQTT, is adopted to ensure the consistency and reliability of data in the transmission process. After receiving the raw data, the monitoring platform performs data cleaning (to remove outliers, noise, etc.) and data aggregation (to integrate the data of multiple data sources together) to provide an accurate and comprehensive view of the resource usage status.
The monitoring platform displays the use state of each resource in real time through a graphical interface, such as a histogram, a line graph, a pie chart and the like. Setting thresholds of resource use (such as processor use rate exceeding 80%), when actual use exceeds the thresholds, automatically triggering an alarm by the system, and informing a manager by means of mail, short message and the like so as to take measures in time to prevent potential problems.
When specifically applied, it is assumed that a data center management system is monitoring a room containing 100 servers. And a temperature sensor and a voltage sensor are deployed on each server, and the temperature and voltage changes of the servers are monitored in real time. The data collector collects data from the sensors every 5 seconds and transmits the data to the database of the monitoring platform via the ethernet. And the data processing module of the monitoring platform cleans and aggregates the received data, and calculates key indexes such as average temperature, highest temperature, voltage fluctuation and the like of each server. The average temperature of each server isWherein, the method comprises the steps of,Representing the number of temperature samples collected during a data collection period; Represent the first And (3) collecting temperature samples. Highest temperature ofWherein, the method comprises the steps of,Represent the firstAnd a temperature sample. The calculation formula of the voltage fluctuation isWherein, the method comprises the steps of,Represent the firstA voltage sample. These indices are presented to the manager in a graphical form via a visual interface. When the temperature of a certain server exceeds a preset threshold (such as 65 ℃), the system automatically sends an alarm mail to a manager to prompt the manager to check whether the cooling system of the server works normally.
In a preferred embodiment of the present invention, the step 12 of collecting and analyzing the current running service and the service demand data about to be online in real time according to the load data of the data center, including the service type, the status, the resource consumption, the expected online time and the expected resource demand, to obtain the service analysis result may include:
Step 122, integrating the real-time business data and the planning data of the on-line business to a unified data platform to obtain an integrated data set;
step 123, cleaning and preprocessing the integrated data set, and extracting key features to obtain a processed data set;
Step 124, analyzing the resource usage pattern of the current running service according to the processed data set, identifying the service type with large resource consumption or fast growth, and marking as the key service;
Step 125, according to the marked key service and service rule, expected income and customer influence factors, evaluating the priority of the service to be on line, and outputting a service priority list;
Step 126, analyzing the influence of the business with different priorities on the resources according to the business priority list and the resource use mode to generate a business resource use analysis report;
and step 127, according to the service resource usage analysis report, obtaining a service analysis result.
In an embodiment of the present invention, real-time business data from different sources (e.g., databases, log files, APIs, etc.) and planning data for an upcoming online business are integrated into a unified data platform through a data extraction, conversion, and loading (ETL) process. The data includes key information such as service type, status, resource consumption, expected online time, and expected resource demand. And cleaning the integrated data set to remove repeated, wrong or incomplete data. Data preprocessing, such as normalization, etc., is performed to eliminate noise and outliers in the data. And extracting key characteristics such as resource consumption modes, growing trends and the like of the service types to form a processed data set.
The processed data set is further analyzed by data analysis techniques (e.g., statistical analysis), and first, resource consumption indicators, such as average values, peak values, etc., of each service type are calculated. The calculation formula of the average value is AWherein, the method comprises the steps of,Representing the total number of acquisitions; Represent the first The resource consumption data of the secondary collection. PWherein, the method comprises the steps of,Represent the firstThe resource consumption data of the secondary collection. And comparing the resource consumption index with a set threshold value, and identifying the service type with large resource consumption or rapid growth. The identified traffic type with large or fast growing resource consumption is marked as critical traffic. And verifying the result of the marked key business to ensure the accuracy and rationality of the marked key business. If the situation of mislabeling or omission is found, the analysis method and threshold setting are adjusted in time, and the analysis process is optimized. And outputting the analysis result in the form of a report, wherein the analysis result comprises the resource consumption condition of each service type, the identification result of the key service and the corresponding analysis chart and description. The priority assessment model is constructed in combination with marked key services, service rules (such as service level agreement SLA, service importance, etc.), expected benefits and customer influencing factors (such as customer satisfaction, market demand, etc.).
And carrying out priority evaluation on the service to be online, and outputting a service priority list to guide resource allocation and online planning. And analyzing the influence of the business with different priorities on the resources according to the business priority list and the resource use mode of the current running business. A business resource usage analysis report is generated by using visualization tools (such as charts, dashboards and the like) to show the resource consumption condition, the growing trend and the expected resource demand of each business type. And obtaining service analysis results including resource optimization suggestions, service adjustment strategies and the like according to the service resource usage analysis report. The analysis results are provided to a decision layer and related teams to support business decisions and resource management.
By analyzing the service data and predicting the resource demand in real time, the resources can be more accurately allocated and adjusted, and the situations of resource waste and deficiency are avoided. Based on the overall business analysis results, the decision layer can make more intelligent business decisions, such as adjusting business strategies, optimizing product combinations, and the like. Customer satisfaction and loyalty can be improved by prioritizing critical services and meeting customer needs, thereby increasing market share and revenue. The real-time monitoring and analysis of the business data can help enterprises to find potential risks and problems in time, and corresponding countermeasures are adopted to reduce business risks. By constructing a unified data platform and analysis flow, enterprises can be promoted to change to data-driven culture, and the overall operation efficiency and innovation capability are improved.
When specifically applied, the above specifically includes:
Data of the current running service is collected in real time, including service type, state, resource consumption (such as CPU, memory, storage, etc.), etc. Planning data for the upcoming service is collected, including service type, projected time to line, projected resource demand, etc. The data is integrated into a unified data platform, such as a data warehouse or a large data lake. And cleaning the integrated data set to remove repeated, wrong or incomplete data.
Key features such as traffic type, resource consumption pattern, expected online time, etc. are extracted for further analysis. And analyzing the resource use mode of the current running service by using a statistical analysis and a machine learning algorithm.
Service types that consume large or fast-growing resources are identified, which often have a significant impact on the stability of the system and are therefore marked as critical services. The priority of the service to be online is evaluated according to marked key service, service rules (such as service importance, emergency degree, etc.), expected benefits and customer influencing factors (such as customer satisfaction, user growth, etc.). A service priority list is output for resource allocation and scheduling. And analyzing the influence of the business with different priorities on the resources according to the business priority list and the resource use mode. A business resource usage analysis report is generated, and the report comprises the contents of resource demand prediction, resource bottleneck identification, resource optimization suggestion and the like. And obtaining a service analysis result according to the service resource usage analysis report. These results can be used to guide decisions on resource allocation, traffic scheduling, system optimization, etc.
Assume that an e-commerce platform is running multiple services including merchandise search, order processing, user recommendation, etc. At the same time, the platform also plans to bring a new promotional program. And collecting operation data of business such as commodity searching, order processing, user recommendation and the like in real time, wherein the operation data comprise business types, states, resource consumption and the like. Planning data for new promotional program businesses is collected, including business type (promotional program), projected time on line (next wednesday), projected resource demand (large amounts of CPU and memory resources), and the like. And cleaning and preprocessing the collected data, and extracting key characteristics such as service types, resource consumption modes and the like. Analysis finds that commodity searching business consumes more resources in peak hours and increases rapidly, and is therefore marked as a key business.
The priority of the new promotional program service is assessed to be high based on key services (merchandise search), service rules (promotional program has an important impact on user experience and sales), expected revenue (promotional program is expected to bring a large sales increase), and customer impact factors (user expects promotional program). The influence of the new sales promotion business on the resources is analyzed, and the business is found to possibly compete with the commodity searching business for a large amount of CPU and memory resources after being on line, so that the resource bottleneck is caused. Generating a service resource usage analysis report, suggesting to increase resource quota in advance or to optimize resource scheduling policy to ensure stable operation of new promotional activity services and critical services. According to the service resource usage analysis report, a service analysis result is obtained, wherein the new sales promotion service has high priority, but the resource competition problem with the commodity searching service is required to be noted. Resource planning and optimization are proposed in advance to ensure stable operation of the service and user experience.
In another preferred embodiment of the present invention, the step 125 of evaluating the priority of the service to be online according to the marked key service and service rule, the expected benefit and the customer influencing factor, and outputting the service priority list may include:
Step 1255, setting evaluation criteria and metrics, including urgency of service;
step 1256, for each service to be online, collecting relevant information including service demand document and customer feedback, and using the relevant information as input data of evaluation standard;
Step 1257, marking the service type identified as having large resource consumption or fast growth as key service information, and grading each service to be online according to the set evaluation standard and the key service information;
step 1258, in the scoring process, assigning a corresponding score according to the matching degree of the business and each evaluation criterion;
step 1259, summarizing the scores of each service to obtain a comprehensive score, and sorting the services according to the comprehensive score, and generating a service priority list from high to low.
In the embodiment of the invention, the urgency of business online is evaluated, such as whether seasonal activities, market response speed requirements and the like are involved. And analyzing economic benefits, market share increase, user liveness improvement and the like possibly brought by the online business. Consider the potential impact of business on customer experience, satisfaction, retention, etc.
The resources (e.g., servers, storage, bandwidth, etc.) required for the service to be online are evaluated for potential impact on existing system resources. For each service to be online, collecting the service demand document of the service to know the specific function, technical requirement and expected target of the service. And carrying out expected benefit analysis, including market research, user research and the like, so as to quantify potential benefits after the business is online. Customer feedback is collected, knowing the user's needs and desires for similar business, and possible market reactions. Based on previous analysis of the resource usage patterns, traffic types that have been identified as having a large or fast-growing consumption of resources are marked as critical traffic. And for each service to be online, scoring item by item according to the set evaluation standard and the key service information. In the scoring process, corresponding scores are given according to the matching degree of the business and each evaluation standard. For example, a business with high urgency, high expected revenue, positive customer impact, and reasonable resource demand will receive a high score. Summarizing the scores of each business to obtain a comprehensive score. And sequencing the services according to the comprehensive scores, and generating a service priority list from high to low.
Through the determined evaluation standard and the determined measurement index, the priority of the service to be online can be rapidly and accurately evaluated, and the decision efficiency is improved. By considering the evaluation criterion of resource demand, system resources can be more effectively allocated and scheduled, and resource waste and bottlenecks are avoided. The influence of the customer is used as one of evaluation criteria, so that the customer requirements can be better met after the business is on line, and the customer satisfaction degree and loyalty degree are improved. By comprehensively considering factors such as urgency of service, expected benefits, resource requirements and the like, the development among different services can be more effectively coordinated, and the overall collaborative development of the services is promoted. By evaluating and analyzing the priority and resource demand of the service to be online, the running condition and resource consumption condition of the service after being online can be predicted better.
For example, in step 1258, for each upcoming service (assumed to be a service) The score under each evaluation criterion can be calculated by the following formula:
;
Wherein, Representing a winter sales; Representing sales in spring; Representing summer sales; Representing autumn sales; Representing a customer feedback score; Representing a sales history growth rate; representing competitor market share changes; indicating a competitor price change; indicating competitor advertising expenditure changes; 、、、、 Representing the coefficients.
When specifically applied, the above specifically includes:
Check if the business is associated with a particular season or holiday, such as a spring festival promotional program, summer travel products, etc. The market demand for the business is evaluated, as well as the competitor's dynamics, to determine if a quick online is needed to preempt the market. The increase of economic indexes such as sales, profits and the like after the business is on line is predicted through market research. And whether new users can be attracted after the business is online is analyzed, so that the market share is enlarged, and the brand awareness is improved. And evaluating the promotion effect of the service on the user activity, such as increasing the access times of the user, improving the residence time of the user and the like. And analyzing improvement or potential problems of the service experience of the clients after the service is online, such as interface friendliness, operation convenience and the like. And evaluating the influence of the service on the customer satisfaction degree and the retention rate, and if the user satisfaction degree is improved, reducing the user loss. Infrastructure resources such as servers, storage, bandwidth, etc. required for the service to be online are detailed and their potential impact on existing system resources is evaluated. And predicting the resource consumption condition of the service after the service is online according to the service characteristics and the historical data so as to be ready in advance. For each upcoming service, its detailed requirements documents are collected, including functional requirements, technical requirements, intended targets, etc. Through market research and user research, potential benefits after business online are quantified, including economic benefits, market share increase, user liveness improvement and the like. The user's needs and desires for similar business, and possibly market reactions, are collected to assess the business's customer acceptance and market potential. And marking the service type with large resource consumption or rapid growth as a key service according to the previous analysis of the resource usage pattern, and taking the service type as an important point of evaluation. And for each service to be online, scoring item by item according to the set evaluation standard and the key service information. In the scoring process, corresponding scores are given according to the matching degree of the business and each evaluation standard. Summarizing the scores of each business to obtain a comprehensive score. And sequencing the services according to the comprehensive scores, and generating a service priority list from high to low.
Suppose an e-commerce platform plans to go online two new businesses, one for the annual goods promotion during the spring festival and the other for the fashion trend commodity special area of the young user. The annual goods sales promotion activity is closely related to the seasonal activity of spring festival, and the market demand is vigorous, so the urgency is high. The fashion trend commodity special area is not limited by seasons, but similar special areas are released in the market, and the demand of the young user group on new products and trend commodities is higher, so that the fashion trend commodity special area has certain urgency. Annual sales promotion is expected to bring about a significant sales increase while increasing the share of the platform in the annual market. Fashion trend commodity special areas are expected to attract more years of light users, improve the activity of the users and the overall income of the platform, and simultaneously enhance the brand influence of the platform in young user groups. Annual goods promotion activities require optimizing the shopping experience of the platform, such as providing convenient payment and logistics services to meet the shopping needs of users during spring festival. The fashion trend commodity special area needs to pay attention to aesthetic and shopping habits of young users, and personalized recommendation and high-quality customer service are provided so as to improve customer satisfaction and retention rate. The promotional program may require an increase in server and bandwidth resources to cope with the proliferation of user access during the spring festival. Fashion trend commodity areas may require more storage resources to store commodity pictures and videos, and more powerful recommendation systems to provide personalized services. And collecting the demand documents of two businesses, and knowing the specific content of the sales promotion, the setting of a commodity special area, the expected target and the like. Through market research and user research, the potential benefits of two businesses after being on line are predicted, including sales increase, market share increase, user activity increase and the like. The user's needs and desires for annual goods promotions and fashion trends commodity areas, as well as possible market reactions, are collected. Annual goods promotion is marked as a critical business based on the analysis of the resource usage patterns because of its large resource consumption and close correlation with the peak of the user.
In a preferred embodiment of the present invention, the step 13 of inputting the load data and the service demand data monitored in real time into the resource allocation prediction and optimization model for training and learning according to the service analysis result, and predicting the resource demand in a future period of time to obtain a final prediction result may include:
Step 133, determining characteristics related to resource demand prediction, including historical load data, service growth trend and service type, according to the service analysis result;
Step 134, constructing a resource allocation prediction and optimization model, and determining input items of the optimization model, including load data, business demand data and output items monitored in real time, including resource demand predicted values in a future period of time;
Step 135, dividing the historical load data and the business demand data into a training set, a verification set and a test set, training the optimization model by using the training set data, and evaluating the prediction performance of the optimization model on the verification set by using root mean square error in the training process to obtain an evaluation result;
136, performing iterative optimization on the optimization model according to the evaluation result, wherein the iterative optimization comprises parameter adjustment;
Step 137, training, verifying and evaluating are repeated until the performance of the optimized model reaches a preset standard, so as to obtain a final optimized model;
And step 138, calculating a resource demand predicted value in a period of time in the future by taking the current real-time monitored load data and the business demand data as input items according to the final optimization model so as to obtain a final predicted result, wherein the final predicted result comprises the resource demand predicted value, the predicted use amount of various types of resources and the growth trend.
In the embodiment of the invention, load data of the system in the past period of time is collected and analyzed, including CPU utilization rate, memory occupation, disk I/O and the like, so as to know the historical consumption mode of system resources. The growth trend of the business, including user growth, transaction amount growth, etc., is analyzed to predict the demand of future business for resources. The difference in demand for resources by different types of traffic, such as computationally intensive, is identified in order to more accurately predict resource demand. The intensive calculation formula is as follows: Wherein, the method comprises the steps of, Representing the total CPU resources required by the intensive business; representing the basic CPU requirements; representing the user growth; Representing CPU consumption per user; representing the transaction amount increment; Indicating CPU consumption per transaction. The total CPU resources required by the system over a period of time are dynamically calculated by taking into account the base CPU requirements, the additional CPU consumption by the user growth, and the additional CPU consumption by the transaction volume growth. This helps system administrators or operators to plan and allocate CPU resources in advance according to the business growth trend and the resource consumption pattern, so as to ensure the stability and performance of the system. Load data (e.g., current CPU usage, memory usage, etc.) and business demand data (e.g., expected user growth, transaction volume, etc.) monitored in real time. Predicted values of resource demand in a future period of time include predicted usage of various types of resources, increasing trend, and the like.
And selecting time sequence analysis to construct an optimization model. The historical load data and the business demand data are divided into a training set, a verification set and a test set. The training set is used for training the model, the verification set is used for evaluating the performance of the optimized model, and the test set is used for finally verifying the model effect. Training the model by using training set data, and enabling the optimization model to accurately predict resource requirements by continuously adjusting model parameters. In the training process, the prediction performance of the model on the verification set is estimated by using an estimation index such as root mean square error (R). If the performance is poor, iterative optimization is performed. And according to the evaluation result, adjusting parameters of the model, such as changing the learning rate, increasing the iteration number and the like, so as to improve the prediction performance of the model. The training, validation and evaluation process is repeated until the model performance reaches a preset criteria, such as R being reduced to an acceptable range. And taking the current load data and the current service demand data which are monitored in real time as input items, and inputting the input items into a final optimization model. The model outputs predicted values of resource demands in a future period of time, including predicted usage amounts, growth trends and the like of various types of resources.
And determining a reasonable resource allocation plan according to the prediction result, so as to ensure that the system can stably operate and meet the service requirements. By accurately predicting the resource demand in a future period of time, the system resources can be more effectively allocated and utilized, and resource waste and shortage are avoided. According to the resource demand prediction result, the configuration of the system and the expansion resources can be timely adjusted, and the system can still keep stable running under high load. Through reasonable resource allocation and optimization, unnecessary resource purchasing and operation cost can be reduced, and the overall economic benefit is improved. The resource demand is accurately predicted and prepared in advance, the service change and the market demand can be responded more quickly, and the service competitiveness is improved.
When specifically applied, the above specifically includes:
System load data is collected over a period of time (e.g., one year), including CPU utilization, memory usage, disk I/O, etc., and corresponding traffic data, such as user growth, transaction growth, etc.
And monitoring load data and business demand data of the current system in real time to be used as the supplement of model input. And cleaning the data to remove abnormal values and missing values. The data are ordered according to a time sequence and divided into a training set, a verification set and a test set. For example, 70% of the data may be used as a training set, 15% of the data as a validation set, and 15% of the data as a test set. According to the characteristics of the data and the prediction requirements, a proper time sequence analysis model such as ARIMA is selected, the model is trained by using training set data, and the model can predict the resource requirements more accurately by continuously adjusting model parameters (such as learning rate, iteration times and the like). In the training process, the prediction performance of the model on the verification set is estimated by using an estimation index such as root mean square error (R). And if the model performance is poor, performing iterative optimization. And according to the evaluation result, adjusting parameters of the model, such as changing the learning rate, increasing the iteration times and the like. The training, validation and evaluation process is repeated until the model performance reaches a preset criteria, such as R being reduced to an acceptable range.
And taking the current load data and business demand data which are monitored in real time as input items, and inputting the input items into the optimized resource allocation prediction and optimization model. The model outputs predicted values of resource demands in a future period (such as one month), including predicted usage of various types of resources, growing trend, and the like. And determining a reasonable resource allocation plan according to the prediction result. For example, if future CPU utilization is predicted to increase substantially, CPU resources need to be increased in advance, and if memory occupancy is predicted to reach a threshold, expansion memory needs to be considered.
Assuming an online shopping platform, the CPU resource requirement of one month in the future needs to be predicted. The following is a specific step of collecting CPU usage data, user growth data, and transaction amount data for the past year.
The data is cleaned and ordered and divided into a training set, a verification set and a test set. The LSTM model was selected for time series analysis. The LSTM model is trained using the training set data, adjusting model parameters to minimize R. And evaluating the performance of the model on the verification set, and performing iterative optimization if the performance is poor. And inputting the current CPU utilization rate, user growth data and transaction amount data monitored in real time into the optimized LSTM model. The model outputs predicted values of CPU resource demands for one month in the future, including average CPU usage per day, peak CPU usage, etc. And determining a resource allocation plan according to the prediction result. For example, if it is predicted that peak CPU usage for a future day will reach 90%, then CPU resources need to be increased in advance to ensure system stability.
In a preferred embodiment of the present invention, the root mean square error is calculated as:
R;
wherein R represents root mean square error; a total number of samples representing the validation set; Represent the first Actual values of the individual samples; Represent the first Predicted values for the individual samples; Represent the first Weights of the samples; Representing regularization coefficients; Representing the number of model parameters; Represent the first And model parameters.
In an embodiment of the invention, sample data of the validation set is collected, including actual valuesAnd predicted valueAnd the weight of each sample. Determining model parametersAnd regularization coefficient. For each sample in the validation set, the square of the difference between the actual and predicted values is calculated, i.e. Multiplying these squared differences by corresponding sample weightsAnd summing to obtain the square sum of the weighted prediction errors: . For each parameter in the model, its square is calculated, i.e Multiplying these sums of squares by a regularization coefficientObtaining regularization term of model parameter. Adding the square sum of the weighted prediction errors and the regularization term of the model parameters to obtain a total error term. Dividing the total error term by the sum of the sample weightsTo obtain an average error. The mean error is square-root-divided to obtain a root mean square error (R).
R not only considers the prediction precision of the model (measured by the square sum of weighted prediction errors), but also introduces a regularization term to punish the complexity of model parameters, thereby avoiding the problem of model overfitting.
By introducing regularization terms, R encourages the model to fit training data while maintaining the simplicity of the parameters, which helps to improve the generalization ability of the model over unknown data. Sample weightAllowing us to flexibly adjust to the importance of different samples. R is an intuitive metric, and smaller values indicate better predictive performance of the model. This allows for a convenient comparison of the predicted performance under different models or different parameter settings.
In a preferred embodiment of the present invention, the calculation formula of the predicted value of the resource demand in the future period of time is:
;
Wherein, Indicating future time of dayResource demand forecast values of (2); An intercept term representing the model; An index representing the influencing factors; Represent the first Weighting of individual influencing factorsRepresent the firstThe time of each influencing factorIs of the observed value of (2)Representing the deviation between the predicted value and the actual value.
In an embodiment of the invention, it is determined that a future time is to be predictedIs to be controlled by the resource requirements of (a). Identifying and selecting 10 key factors affecting future resource requirements(=1, 2,..10), These factors are historical resource usage, economic conditions, population changes, etc. Time series data is collected, including historical resource demands and time series values for various influencing factors. And cleaning the data, and processing the missing value and the abnormal value to ensure the integrity and the accuracy of the data. And (5) carrying out standardization or normalization treatment on the data according to the requirement. Based on the predicted objective and the influencing factors. In the case of this model of the present invention,Is a function of the variable quantity, and is,Is the intercept term of the term,Is the firstThe weight of the individual influencing factors is determined,Is the firstThe time of each influencing factorIs used for the observation of the (a),Is the deviation between the predicted value and the actual value. The model is represented in a matrix form. Is provided withFor a resource demand vector (containing historical data and placeholders for future points in time that need to be predicted),To influence the factor matrix (comprising intercept term and time series values of 10 influencing factors),For the parameter vector to be estimated (comprising intercept term and 10 weights),Is an error vector. Estimating a parameter vector by minimizing the sum of squares of the deviations between the predicted and actual values. First, a factor matrix is calculatedIs transposed of (a)T. The equation is: Obtaining a parameter vector according to the equation Is used for the estimation of the estimated value of (a). Using estimated parameter vectorsAnd future influencing factor valuesHere, theRefers to the value of the future time point, and substitutes the value into the model formulaCalculating future timeResource demand forecast values of (2)。
By comprehensively considering a plurality of influencing factors and utilizing historical data to carry out parameter estimation, the accuracy of resource demand prediction can be improved. Accurate resource demand prediction helps decision makers such as enterprises and governments determine reasonable resource allocation plans, and avoids resource waste or shortage. According to the prediction result, the configuration and the use of resources can be optimized, the utilization efficiency of the resources is improved, and the cost is reduced. The predictive model may help identify potential resource risks, such as supply interruptions, demand surges, etc., to take action in advance for risk management. Accurate resource demand predictions help balance the relationship of economic development with resource protection.
In a preferred embodiment of the present invention, the step 14 dynamically allocates and optimizes the resources of the data center according to the final prediction result, including adjusting the number of processor cores, memory allocation, and network bandwidth allocation, may include:
And dynamically adjusting the allocation of the processor core number according to the result of the prediction model. For example, during periods of predicted high load, the number of processor cores allocated to a particular application may be increased to ensure performance. Tasks are evenly distributed to a plurality of processor cores through a load balancing algorithm, so that overload of certain cores and idle of other cores are avoided. And dynamically adjusting memory allocation according to the memory demand prediction of the application. For memory intensive applications, more memory resources may be allocated. And periodically recovering the memory resources which are not used any more and reallocating the memory resources to the needed application so as to improve the memory utilization rate. And predicting future network traffic by using the prediction model so as to adjust network bandwidth allocation in time. The bandwidth priority is set for different applications, so that the key application can still obtain enough bandwidth when the network is congested. And through a data center management system, key indexes such as the utilization rate of the processor, the memory occupation, the network bandwidth consumption and the like are monitored in real time. And dynamically adjusting the resource allocation according to the real-time monitoring data and the prediction result. For example, when it is detected that the actual resource demand of an application exceeds a predicted value, the resources allocated to the application may be immediately increased. And establishing a feedback mechanism, and collecting feedback comments of the user on the resource allocation effect. These feedback can be used to improve the predictive model and dynamic allocation strategy. The predictive model and dynamic allocation policies are evaluated and optimized periodically to accommodate changes in data center resource requirements.
When specifically applied, the above specifically includes:
The tasks are evenly distributed across multiple processor cores using a load balancing algorithm, such as polling, to avoid overloading some cores while others are idle. Specifically, the number of processor cores available in the system is obtained, which may be accomplished through an API provided by the operating system. A poll pointer is set to track the processor cores that are currently assigned tasks. The initial value is set to 0. A task queue is created for storing tasks to be allocated. The tasks may be computing tasks, I/O operations, or other operations requiring processor resources. And taking out a task from the task queue. Tasks are assigned to corresponding processor cores based on the current value of the poll pointer. The polling pointer is updated to point to the next processor core. If the polling pointer has reached the maximum of the number of cores, it is reset to 0, forming a loop. The tasks are submitted to the assigned processor cores for execution.
If the system supports multiple threads or processes, the corresponding mechanisms may be used to bind tasks to the designated cores for execution. During task execution, the load condition of each processor core is monitored. If a core is found to be overloaded or idle for too long, the polling strategy may be dynamically adjusted, such as increasing or decreasing the amount of tasks allocated to the core.
Suppose that some online game is rushing in user access volume from 8 to 10 pm on weekends, resulting in processor overload. Through the prediction model, an administrator increases the number of processor cores for the game in advance, and configures a load balancing strategy to ensure smooth running of the game.
And predicting future memory demands according to the historical memory use condition and business growth trend of the application. More memory resources are allocated for the memory-intensive applications, ensuring that they have sufficient memory available during operation.
And periodically scanning the use condition of the memory, recovering the memory resources which are not used any more, and reallocating the memory resources to the needed application, thereby improving the utilization rate of the memory.
For example, a big data analysis platform consumes a lot of memory when processing a lot of data. And an administrator allocates enough memory resources for the platform in advance according to the prediction model, and sets a memory recovery mechanism to ensure the efficient utilization of the memory. And predicting future network traffic, particularly traffic during peak hours and emergencies, by using the prediction model. Higher bandwidth priority is set for critical applications, ensuring that adequate bandwidth resources are still available when the network is congested. According to the real-time flow data and the prediction result, the network bandwidth allocation is dynamically adjusted, and the bandwidth waste and bottleneck phenomenon are avoided. For example, in a large online campaign, it is predicted that the amount of web site access will increase substantially and the network bandwidth requirements will proliferate. Administrators have increased allocation of network bandwidth in advance and set high priority for critical traffic, ensuring that the network is unobstructed during activity.
The data center management system is used for monitoring key indexes such as the utilization rate of a processor, the occupation of memory, the consumption of network bandwidth and the like in real time. And setting an alarm mechanism, and timely notifying an administrator to process when a certain index exceeds a preset threshold value. And dynamically adjusting the resource allocation according to the real-time monitoring data and the prediction result. For example, when it is detected that the actual resource demand of an application exceeds the predicted value, the resources allocated to the application are immediately increased, so that the normal operation of the application is ensured. And the automatic script is used for realizing the rapid adjustment of the resources, so that the efficiency and the accuracy of resource allocation are improved. And collecting feedback opinion of the user on the resource allocation effect through the modes of user investigation, system logs and the like.
User feedback is analyzed periodically to identify problems and improvement points in the allocation of resources. And continuously optimizing a prediction model and a dynamic allocation strategy according to the user feedback and the real-time monitoring data. For example, adjusting parameters of the predictive model, improving load balancing algorithms, etc. And periodically evaluating and planning the data center resources to ensure that the resources can meet the requirements of future business development. Meanwhile, the resource allocation strategy is timely adjusted according to the service change, and the resource utilization rate and the service response speed are improved.
In a preferred embodiment of the present invention, the step 15 automatically performs adjustment of corresponding resource allocation according to dynamic allocation and optimization of resources of the data center, and continuously monitors an operation state of the data center to obtain new load data and service demand data in real time, and may include:
step 155, receiving the final prediction result and automatically triggering a resource allocation adjustment flow;
Step 156, determining a corresponding resource allocation adjustment scheme according to the content of the prediction result, including increasing or decreasing server resources, adjusting network bandwidth allocation or reallocating storage resources;
Step 157, sending the resource allocation adjustment scheme to a management system of the data center, and performing a resource allocation adjustment operation;
Step 158, continuously monitoring the operation state of the data center, including key indexes such as processor utilization, memory occupancy, network bandwidth utilization, and use condition of storage devices, so as to obtain new load data and service demand data in real time, including real-time traffic, user request amount, and transaction processing amount.
In the embodiment of the invention, according to the prediction result, the system intelligently determines the corresponding resource configuration adjustment scheme. These schemes include increasing or decreasing server resources, adjusting network bandwidth configurations, reallocating storage resources, and the like. In determining a solution, the system may consider a number of factors, such as cost effectiveness, resource utilization, business continuity, etc., to ensure the rationality and feasibility of the solution. The system sends the resource configuration adjustment scheme to a management system of the data center. After receiving the instruction, the data center management system automatically executes the resource allocation adjustment operation. These operations involve the start-up and shut-down of the server, reallocation of network bandwidth, partitioning of storage resources, and the like. The operation state of the data center is continuously monitored, and the operation state comprises key indexes such as processor utilization rate, memory occupancy rate, network bandwidth utilization rate, use condition of storage equipment and the like. These metrics are obtained in real time by the data center management system. The collected monitoring data is analyzed by the system in real time to identify any anomalies or trends. These data may also be used to update the prediction model to improve the accuracy of future predictions. And the system dynamically adjusts the resource allocation strategy according to the monitoring result and the service demand data.
By dynamically allocating and optimizing the resources of the data center, the system can ensure that the resources are fully utilized when needed, and avoid waste. This helps to reduce the operating costs of the data center and to increase overall efficiency. The system automatically monitors and adjusts the resource allocation, and can ensure that the data center keeps running stably when the service demand changes. This helps to reduce the risk of service interruption due to insufficient or overloaded resources. By adjusting the resource allocation in real time, the system can ensure that the data center always provides sufficient processing power, network bandwidth and storage resources to meet user demands. This helps to improve the user experience and satisfaction. The automatic resource allocation adjustment flow simplifies the management work of the data center. The system can automatically complete the resource allocation and adjustment tasks without manual intervention by an administrator. This helps to reduce the management cost and improve the management efficiency. The system can dynamically adjust the resource allocation according to the change of the service demand, and provides powerful support for service growth.
When specifically applied, the above specifically includes:
The system carries out deep analysis on the prediction result to know the demand trend of the data center resource in a future period of time. The prediction results comprise expected changes of key indexes such as processor utilization, memory occupancy, network bandwidth utilization and use condition of storage equipment. Based on the prediction result, the system intelligently determines the corresponding resource configuration adjustment scheme. The solution determines the number of server resources that need to be increased or decreased, the adjustment range of the network bandwidth, the reallocation mode of the storage resources, and the like. In determining the solution, the system may comprehensively consider various factors, such as cost effectiveness (to ensure that the adjustment solution is economically reasonable), resource utilization (to improve resource utilization efficiency and avoid waste), service continuity (to ensure that the adjustment process does not affect service operation), and the like. The system sends the resource configuration adjustment scheme to a management system of the data center. After receiving the instruction, the data center management system automatically executes the resource allocation adjustment operation. These operations involve start-up and shut-down of servers (increasing or decreasing the number of servers according to demand), reallocation of network bandwidth (adjusting the allocation ratio of network bandwidth to meet the demands of different services), partitioning of storage resources (repartitioning or expanding storage devices), etc. The system can continuously monitor the running state of the data center, including key indexes such as the utilization rate of a processor, the occupancy rate of a memory, the utilization rate of network bandwidth, the use condition of storage equipment and the like.
The collected monitoring data is analyzed by the system in real time to identify any anomalies or trends. If an abnormality or trend is found, the system can give an alarm in time, and the resource allocation strategy is dynamically adjusted according to the monitoring result and the business demand data.
Assuming that a data center predicts a future week, network bandwidth requirements will increase substantially and storage resource requirements will decrease as new traffic comes on-line. According to the prediction result, the system determines the following resource allocation adjustment scheme, namely, increasing the allocation proportion of the network bandwidth and ensuring that the new service can run smoothly. The method is particularly operated to reallocate part of network bandwidth originally allocated to other services to new services to be brought online so as to meet the high bandwidth requirements of the new services. And re-partitioning the storage device to release part of the unused storage space for other business requirements. Meanwhile, storage resources are optimized, and storage efficiency is improved.
After the adjustment operation is performed, the operating state of the data center is continuously monitored. If abnormal fluctuation of network bandwidth or storage resources is found, the system immediately analyzes and dynamically adjusts the resource allocation strategy according to actual conditions. For example, if the network bandwidth actually used by the new service is lower than the predicted value, the system may reallocate part of the network bandwidth to other services, and if the storage resource is in tension, the system may consider adding storage devices or optimizing storage strategies.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the present invention.
Claims (2)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411570176.0A CN119094335B (en) | 2024-11-06 | 2024-11-06 | A method for automatically configuring data center resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411570176.0A CN119094335B (en) | 2024-11-06 | 2024-11-06 | A method for automatically configuring data center resources |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119094335A CN119094335A (en) | 2024-12-06 |
| CN119094335B true CN119094335B (en) | 2025-03-14 |
Family
ID=93663010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411570176.0A Active CN119094335B (en) | 2024-11-06 | 2024-11-06 | A method for automatically configuring data center resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119094335B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119782187B (en) * | 2025-01-03 | 2025-07-18 | 阿内咖咖(无锡)信息技术有限公司 | A multivariable real-time measurement and status testing method for data centers |
| CN119440740A (en) * | 2025-01-08 | 2025-02-14 | 北京庭宇科技有限公司 | Method and system for adaptive elastic expansion strategy of Android distributed edge container |
| CN119766657B (en) * | 2025-03-05 | 2025-05-09 | 顺通信息技术科技(大连)有限公司 | Self-adaptive bandwidth allocation method based on network traffic prediction |
| CN119847454A (en) * | 2025-03-21 | 2025-04-18 | 华能信息技术有限公司 | Dynamic adjustment method and system for storage resource pool |
| CN120743508A (en) * | 2025-06-20 | 2025-10-03 | 首都信息科技发展有限公司 | Big data resource capacity management method and device |
| CN120353609B (en) * | 2025-06-24 | 2025-09-09 | 南昌首页科技股份有限公司 | Elastic demand calculation method and application system for server transaction |
| CN120371544B (en) * | 2025-06-27 | 2025-09-26 | 天津津能电力科学研究有限公司 | Dynamic allocation method for data processing resources of data center station |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024212614A1 (en) * | 2023-12-13 | 2024-10-17 | 天翼云科技有限公司 | Hybrid elastic scaling method based on multi-dimensional resource prediction |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110096349B (en) * | 2019-04-10 | 2020-03-06 | 山东科技大学 | Job scheduling method based on cluster node load state prediction |
| CN117762644A (en) * | 2024-01-08 | 2024-03-26 | 黄冈师范学院 | Resource dynamic scheduling technology for distributed cloud computing systems |
| CN118093047B (en) * | 2024-02-02 | 2025-02-11 | 联通沃音乐文化有限公司 | A method for a digital twin XR low-code development platform |
| CN118550711B (en) * | 2024-07-29 | 2024-11-05 | 广脉科技股份有限公司 | A method and system for improving computing power performance |
-
2024
- 2024-11-06 CN CN202411570176.0A patent/CN119094335B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024212614A1 (en) * | 2023-12-13 | 2024-10-17 | 天翼云科技有限公司 | Hybrid elastic scaling method based on multi-dimensional resource prediction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119094335A (en) | 2024-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN119094335B (en) | A method for automatically configuring data center resources | |
| US10896203B2 (en) | Digital analytics system | |
| CN118363765A (en) | Cloud resource automatic allocation system | |
| CN118628219B (en) | IT equipment time-sharing leasing service management system | |
| CN118154233A (en) | Automobile merchant intelligent risk identification system based on merchant state self-monitoring | |
| CN118469405B (en) | Processing method, device, equipment and storage medium for household supply chain data | |
| US8214327B2 (en) | Optimization and staging method and system | |
| CN118467186B (en) | Multi-tenant server-free platform resource management method and system | |
| CN110689217B (en) | Intelligent manufacturing system for computer production | |
| CN110516155A (en) | Method and system for generating marketing strategy | |
| CN118502918A (en) | Workflow intelligent management system based on artificial intelligence | |
| CN119883580B (en) | Task scheduling optimization analysis method and system based on big data | |
| CN117634854A (en) | A multi-cloud management system and method for real-time synchronous monitoring of cloud platform status | |
| CN118014586A (en) | Customer hierarchical classification management system based on big data | |
| CN117876059A (en) | Method, device, equipment and medium for online order management | |
| Park et al. | Combining genetic algorithms and bayesian neural networks for resource usage prediction in multi-tenant container environments | |
| CN120336038A (en) | A data processing method, device, medium and product for a high-performance computing environment | |
| CN119941147A (en) | A project management system and management method | |
| CN119421186A (en) | A method for intelligent dynamic adjustment of flow data | |
| CN118096246A (en) | Enterprise portrait system and method | |
| CN117670549A (en) | Dynamic optimization and intelligent scheduling method of financial assets based on multi-dimensional data | |
| CN117421659A (en) | Charging station monitoring management method, system, terminal equipment and storage medium | |
| CN119558895A (en) | A method and system for optimizing fabric inventory | |
| CN113762695A (en) | Task list distribution method and device | |
| JP2004094662A (en) | Optimization model application method and device for credit risk management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |