TWI611301B - Resource utilization analysis and prediction system and method for cloud environment - Google Patents

Description

Resource utilization analysis and prediction system and method for cloud environment

The invention relates to a resource utilization rate analysis and prediction system and method, in particular to analyze and predict using a small amount or a large amount of historical performance data, and predetermine whether to automatically expand the resource utilization analysis of the cloud environment of the virtual machine resource through the prediction result. Forecasting systems and methods.

With the cloud technology becoming more and more mature, more and more large cloud providers provide cloud services, such as renting virtual machines for customers. However, behind the huge services, how to effectively manage huge computing resources is very important. How to make the user use smoothly and not waste computing resources is an extremely problem to be solved. In the past, when virtual resources were insufficient, the virtual machine could dynamically increase the resources of the virtual machine through the auto-scaling technology to maintain the user's service quality. However, if the resource reaches the upper limit, the adjustment will be performed. Often the quality of the user's service has declined, resulting in a low user evaluation of the service. If the automatic expansion is too early, the computing resources will be wasted. Therefore, if the performance trend of the virtual machine can be predicted in advance, it can be automatically expanded as soon as possible. No waste of resources.

It can be seen that there are still many shortcomings in the above-mentioned methods of use, and there is still room for improvement. To be improved.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventors have improved and innovated, and after years of painstaking research, finally succeeded in research and development of a resource utilization analysis and prediction system and method for a cloud environment of the present invention.

The present invention provides a resource usage analysis and prediction system for a cloud environment, comprising: a setting module, wherein the setting module provides an input interface, and uses the input interface to set a setting parameter, wherein the setting parameter is a prediction time a short-term analysis of the historical performance quantity, the number of time blocks, a long-term analysis historical performance quantity, a monitoring interval, a monitoring resource item, an automatic expansion threshold and an automatic reduction threshold, a performance collection module, the performance The collection module receives the setting parameter of the setting module, and periodically obtains data of the monitoring resource item according to the monitoring interval and the monitoring resource item in the setting parameter, and stores, and a performance prediction module, the performance The prediction module performs a short-term performance analysis prediction and a long-term performance analysis prediction according to the data of the monitoring resource item stored in the performance collection module, and a dynamic resource adjustment module, wherein the dynamic resource adjustment module is based on the performance The short-term performance analysis prediction of the prediction module and the prediction of the long-term performance analysis Automatic expansion of resources or reduction of resources.

The performance prediction module further includes a short-term performance analysis prediction module and a long-term performance analysis prediction module, wherein the short-term performance analysis prediction module uses the short-term performance analysis to predict and calculate the change rate of the performance data during the prediction time to push Estimating the effectiveness of the prediction time, and the long-term performance analysis prediction module uses the long-term performance analysis to predict the performance data in a specific interval by the clustering algorithm, and extracts a plurality of performance data representing the specific interval to estimate The predicted value of future resources.

The dynamic resource adjustment module further includes a determination module, an automatic expansion module and an automatic reduction module, wherein the determination module automatically determines whether the dynamic is based on the performance prediction of the setting module and the performance prediction module. The resource is adjusted and the automatic expansion module is instructed to automatically expand the resource and the automatic reduction module automatically reduces the resources by itself.

The present invention provides a resource usage analysis and prediction method for a cloud environment, the method comprising: setting a parameter for predicting a prediction target through one input interface of a setting module; and collecting, by the performance collection module, the prediction target according to the setting parameter Performance data; a performance prediction module based on the set parameters and the performance data of the forecast target through a short-term performance analysis forecast module to calculate the rate of change of the performance data to estimate the performance, and through a long-term performance analysis prediction module by clustering The algorithm groups the performance data in a specific interval, and extracts a plurality of performance data representing the specific interval to estimate the predicted value of the future resource; and a dynamic resource adjustment module according to the setting module according to the determination module With the performance prediction of the performance prediction module, it is automatically determined whether the resource is dynamically adjusted and an automatic expansion module is commanded to automatically expand the resource and an automatic reduction module automatically reduces the resource by itself.

The resource utilization analysis and prediction system and method for the cloud environment provided by the invention can dynamically adjust cloud resources and accurately allocate cloud resources by predicting future performance trends, so that the user's service maintains certain performance.

The short-term performance analysis prediction and the long-term performance analysis prediction of the present invention can be automatically predicted using different methods according to different situations or times. When the predicted target performance change is large, the short-term performance analysis can be used to predict the future performance by analyzing the rate of change at different times. When the target performance is predicted to be more regular, the long-term performance analysis and prediction can be used to remove the biased performance data through the clustering technique, and the representative group can be retained to calculate the prediction performance. In addition, the short-term performance analysis prediction and the long-term performance analysis prediction Can be applied to forecasting targets at the same time, providing a more robust forecasting method to ensure the quality of user services.

Furthermore, the short-term performance analysis prediction of the present invention effectively estimates the future performance trend by analyzing the rate of change of performance at different time points and giving different weights of changes at different times. Long-term performance analysis predicts the selection of multiple representative groups for statistics, and the predicted performance is less extreme and has higher accuracy.

100‧‧‧ resource pool

101‧‧‧Network

102‧‧‧ physical machine

103‧‧‧ physical machine

104‧‧‧ physical machine

105‧‧‧Virtual Machine

106‧‧‧Virtual Machine

107‧‧‧Virtual Machine

108‧‧‧Virtual Machine

109‧‧‧Virtual Machine

110‧‧‧Virtual Machine

200‧‧‧Resource Usage Analysis and Prediction System for Cloud Environment

210‧‧‧Setting module

220‧‧‧Efficient collection module

230‧‧‧Performance Prediction Module

240‧‧‧Dynamic Resource Adjustment Module

231‧‧‧Short-term performance analysis prediction module

232‧‧‧Long-term performance analysis and prediction module

241‧‧‧Judgement module

242‧‧‧Automatic expansion module

243‧‧‧Automatic reduction module

S510~S560‧‧‧Step process

FIG. 1 is an environment configuration diagram of a resource usage analysis and prediction system of a cloud environment according to the present invention.

2 is an architectural diagram of a resource usage analysis and prediction system for a cloud environment of the present invention.

FIG. 3 is an architectural diagram of the performance prediction module of the present invention.

4 is an architectural diagram of a dynamic resource adjustment module of the present invention.

FIG. 5 is a flow chart of the architecture diagram of the resource usage analysis and prediction system of the cloud environment of the present invention.

Figure 6 is a schematic diagram showing the results of the short-term performance analysis and prediction process of the present invention.

FIG. 7 is a schematic diagram showing the results of the short-term performance analysis and prediction process of the present invention.

FIG. 8 is a schematic diagram showing the results of the long-term performance analysis and prediction process of the present invention.

Hereinafter, a multilingual speech recognition apparatus according to the present invention will be described with reference to related drawings. For the sake of understanding, the same components in the following embodiments are denoted by the same reference numerals.

Referring to FIG. 1, the environment configuration diagram of the resource usage analysis and prediction system of the cloud environment of the present invention is shown in the figure. The resource pool 100 includes a physical machine 102, a physical machine 103, and a physical machine 104, and is connected through the network 101. . The operating systems of the physical machine 102, the physical machine 103, and the physical machine 104 may be a virtualized infrastructure for Citrix XenServer, KVM, or VMware ESXi, and the virtual machines are respectively executed on the physical machine 102, the physical machine 103, and the physical machine 104. 105. Virtual machine 106, virtual machine 107, virtual machine 108, virtual machine 109, and virtual machine 110. Taking XenServer as an example, multiple XenServers can form a resource pool, and resources in the resource pool 100 can be shared. When the virtual machine is executed by the physical machine 102 in the resource pool 100, if the central processor resources of the physical machine 102 are insufficient, The virtual machine can be migrated to the physical machine 103 with sufficient central processor resources by means of instant migration. In addition, if the virtual machine resources are insufficient, but the physical machine resources are still sufficient, the virtual machine resources can be automatically expanded through the cloud technology.

Referring to FIG. 2, the architecture diagram of the resource usage analysis and prediction system of the cloud environment of the present invention is shown in the figure. The resource usage analysis and prediction system 200 of the cloud environment includes a setting module 210 and a performance collection module 220. The performance prediction module 230 and the dynamic resource adjustment module 240. The resource usage analysis and prediction system 200 of the cloud environment may be configured to execute on any physical machine, virtual machine, or a machine connected to a resource pool outside the resource pool. The setting module 210 provides an input interface, allowing the user to input the predicted time, the short-term performance analysis to predict the historical performance quantity used, the number of time blocks, the historical performance quantity used for long-term performance analysis, the monitoring interval, the monitoring resource item, and the automatic expansion. Threshold value and automatic reduction threshold. Monitoring resource projects can be medium Central processor usage, memory usage, hard disk write rate, hard disk read speed, network receive speed, network transfer speed, session usage, connection usage, etc., but not limited to these items . An automatic expansion threshold is set for each resource item, and the automatic expansion threshold is used by the execution dynamic resource adjustment module 240 to determine whether an action needs to be performed. When the predicted value of the performance prediction module 230 exceeds the automatic expansion threshold, the execution dynamic is performed. The operation of the resource adjustment module 240. In addition to the automatic expansion threshold, an automatic reduction threshold is also input for each resource item. The automatic reduction threshold is used by the execution dynamic resource adjustment module 240 to determine whether an action needs to be performed. When the current performance usage is lower than the automatic reduction threshold At this time, the action of the dynamic resource adjustment module 240 is executed. The performance collection module 220 receives the parameters of the setting module 210, monitors the interval and monitors the resource items, and periodically acquires the data of the monitoring resource items, and stores them for use by the performance prediction module 230. The way to monitor can be achieved through Nagios or other existing tools.

Please refer to FIG. 3 , which is an architectural diagram of the performance prediction module of the present invention. The performance prediction module 230 includes a short-term performance analysis prediction module 231 and a long-term performance analysis prediction module 232 , and short-term performance analysis and prediction. The module 231 receives the parameters of the setting module 210 and the performance information of the performance collecting module 220. The parameters of the setting module 210 include the predicted time, the historical performance quantity used for the short-term performance analysis prediction, the monitored item, the monitoring interval, the automatic expansion threshold value and the automatic reduction threshold value of each monitoring item.

Analysis of the predicted short-term performance prediction module 231 is a resource embodiment, PredictByRecentData (t, t ', h, n, U c), wherein

U _c ={u _{c , t} , u _{c , th} , u _{c , t-2h} , u _{c , t-3h} , ... , u _{c , t-nh} , ...}

d _{c , t ', k} =t ' -t+kh

其中，C為監控項目類別的集合，c為資源，t為目前時間，t'為預測的時間點，h為每隔h分鐘蒐集一次效能資訊，n為使用的歷史效能數量，也就是目前時間往前n筆為使用的範圍，U_c為資源c的歷史效能資料的集合，u_c,t為時間t時的資源c的效能，d_c,t',k為預測時間t'與以目前時間為基準過去第k筆效能資訊產生的時間相差的值，w_c,n,k為資源c在過去歷史效能資料中的第k筆的權重，r_c,t為時間t時的資源c的使用率變化量。

Where C is the set of monitoring item categories, c is the resource, t is the current time, t ' is the predicted time point, h is the performance information collected every hour, n is the historical performance quantity used, that is, the current time The forward n is the range of use, U _c is the set of historical performance data of resource c, u _{c , t} is the performance of resource c at time t, d _{c , t ', k} is the predicted time t ' and current The time is the value of the time difference generated by the _k-th performance information in the past, w _{c , n , k} is the weight of the _kth of the resource c in the past historical performance data, and r _{c , t} is the resource c of the time t The amount of usage change.

群的中心點來計算平均做為預測的效能v_average,c，其中v_average,c代 表資源c預測的效能；以及

，其中centroid_i,c為資源c，群i的中心點。 The long-term performance analysis prediction module 232 receives the setting parameters of the setting module 210 and the performance information of the performance collecting module 220. The parameters of the setting module 210 include the predicted time, the number of time blocks, the historical performance quantity used for long-term performance analysis, the monitored items, the monitoring interval, the automatic expansion threshold value and the automatic reduction threshold value of each monitoring item. The prediction step of the long-term performance analysis prediction module 232 is as follows: divide the time of day into b blocks, and if b=24, the day can be divided into 00:00:00~00:59:99, 01:00:00 ~01:59:59, 23:00:00~23:59:59 twenty-four blocks; take out the historical performance data of the block where the time is to be predicted; each monitoring data is used as a vector v _{i , c} , where v _{i , c} is the amount of resource c used at time i; the historical performance quantity e used for long-term performance analysis, grouping all the vectors of the interval in e days, and the grouping method is k-means, and k group is obtained, wherein The similarity calculation method can adopt Euclidean Distance; remove the group whose size is ranked in the latter half. Because these groups are small, they are less representative of the performance of this interval, so before use

The center point of the group to calculate the average as the predicted performance v _{average , c} , where v _{average , c} represents the performance of the resource c prediction;

, where centroid _{i , c} is the resource c, the center point of group i.

Please refer to FIG. 4 , which is a structural diagram of the dynamic resource adjustment module of the present invention. The dynamic resource adjustment module 240 includes three modules, namely, a determination module 241, an automatic expansion module 242, and an automatic reduction. Module 243. The judging module 241 receives the prediction result of the performance prediction module 230 and the current resource usage rate, and determines whether the predicted performance exceeds the automatic expansion threshold or whether the current resource usage rate is lower than the automatic reduction threshold. The expansion module 242 is lower than the automatic reduction module 243. The automatic expansion module 242 can increase the number of virtual machine resources allocated. For example, the virtual machine is originally configured with a core CPU, and can be allocated as a two-core central processor after execution. Conversely, the auto-reduction module 243 can reduce the resources allocated by the virtual machine.

Please refer to FIG. 5 , which is a flowchart of an architecture diagram of a resource usage analysis and prediction system for a cloud environment according to the present invention, which is respectively described by short-term performance analysis prediction and long-term performance analysis prediction.

，則執行長期效能分析預測，兩者可同時執行， 這邊假設執行短期效能分析預測；步驟S530啟動效能蒐集模組，並每隔h分鐘執行一次，每個監控項目的效能資料集合為U_c，c為監控項目或資源，如中央處理器使用率；步驟S540執行短期效能分析預測時，分析效能資料，將步驟S530的效能資料U_c和使用者想要預測的時間t'和使用的效能資料歷史範圍n作為輸入，目前時間t，帶入到

舉例如下：目前時間為17：00，中央處理器使用率50%，預測的時間點為17：10，每2分鐘蒐集一次效能資訊，過去的中央處理器效能資訊，16：48 10%、16：50 20%、16：52 30%、16：54 35%、16：56 20%、16：58 45%，此時PredictByRecentData(t,t',h,n,U _c )=106.32，如下表1及圖6所示：

又一舉例如下：目前時間為17：00，中央處理器使用率20%，預測的時間點為17：10，每2分鐘蒐集一次效能資訊，過去的中央處理器效能資訊，16：48 10%、16：50 40%、16：52 20%、16：54 10%、16：56 30%、16：58 25%，PredictByRecentData(t,t',h,n,U _c )=14.12，如下表2及圖7所示：

步驟S560判斷預測結果是否超過T1或目前監控項目使用率低於T2，若超過則執行自動擴展模組。 Short-term performance analysis and prediction process: Step S510: Let the user input the predicted time t ′ through the setting module, the historical performance quantity used for the short-term performance analysis prediction, the monitoring interval h, the set C of the monitoring item category, and the automatic expansion threshold T1 And automatically reducing the threshold value T2, where T1={t _{1 , 1} , t _{1 , 2} , ... , t _{1 , S} }, T2={t _{2 , 1} , t _{2 , 2} , ... , t _{2 , S} }, t _{1 , 1} is a threshold for automatic expansion of the CPU usage and t _{2 , 1} is a threshold for reducing the resource usage of the CPU, s=|C|; and performing steps S520 and S530 simultaneously, Step S520 determines whether short-term performance analysis prediction or long-term performance analysis prediction should be performed. When current time t, if t%h=0, perform short-term performance analysis prediction, if

Long-term performance analysis and prediction is performed, both can be performed at the same time, here we assumed that the implementation of short-term performance analysis and forecasting; step S530 to start collecting module performance, and perform a h every minute, each monitoring project performance data collection for the U _c , c is a monitoring project or resource, such as central processor usage; when performing short-term performance analysis and prediction in step S540, analyzing performance data, the performance data U _{c of} step S530 and the time t ' and the performance of the user want to predict Data history range n as input, current time t, brought into

For example, the current time is 17:00, the CPU usage is 50%, and the predicted time is 17:10. The performance information is collected every 2 minutes. The past CPU performance information, 16:48 10%, 16 :50 20%, 16:52 30 %, 16:54 35%, 16:56 20%, 16:58 45%, at this time PredictByRecentData ( t,t',h,n,U _c )=106.32, as shown in the following table 1 and Figure 6:

Another example is as follows: the current time is 17:00, the CPU usage is 20%, and the predicted time is 17:10. The performance information is collected every 2 minutes. The past CPU performance information, 16:48 10% , 16:50 40%, 16:52 20%, 16:54 10 %, 16:56 30%, 16:58 25%, PredictByRecentData ( t,t',h,n,U _c )=14.12, as shown in the following table 2 and Figure 7:

Step S560 determines whether the prediction result exceeds T1 or the current monitoring item usage rate is lower than T2, and if it exceeds, executes the automatic expansion module.

群的中心點來計算平均做為預測的效能。 舉例如下：目前時間為16：50，預測的時間點為17：00，每10分鐘蒐集一次效能資訊，上個月過去的中央處理器效能資訊，總共有180筆，中央處理器使用率分別為0%~10% 10筆，10%~20% 15筆，20%~30% 20筆，30%~40% 65筆，40%~50% 30筆，50%~60% 15筆，60%~70% 10筆，70%~80% 5筆，80%~90% 5筆，90%~100% 5筆，100% 0筆，完整歷史效能請參照附錄，利用k-means對這些歷史效能做分群，k=5，，如下表3及圖8所示：

其中，群2、群1、和群5大小為前

群，統計這三個群的平均，因此v_average=42.7，42.7即為中央處理器使用率預測值；步驟S560判斷預測結果是否超過T1或目前監控項目使用率低於T2，若超過則執行自動擴展模組。 Long-term performance analysis and prediction process: Step S510 allows the user to input the predicted time t ' , the number of time blocks b, the historical performance quantity e used for long-term performance analysis, the monitoring interval h, the monitoring resource item C, and the automatic expansion threshold through the setting module. The value T1 and the automatic reduction threshold T2, where T1={t _{1 , 1} , t _{1 , 2} , ... , t _{1 , S} }, T2={t _{2 , 1} , t _{2 , 2} , ... , t _{2 , S} }, t _{1 , 1} is the threshold for the automatic expansion of the CPU usage and t _{2 , 1} is the threshold of the CPU usage reduction, s=|C|; and step S520 and steps are simultaneously performed S530, step S520 determines that the long-term performance analysis and prediction should be performed; step S530 starts the performance collection module, and executes once every h minutes, and the data collected by the resource c at different time points is used as a vector v _{i , c} , where v _{i , c} is the amount of resource c used at time i; the historical performance quantity e used for long-term performance analysis, grouping all the vectors of the interval in e days, assuming the number of time blocks b=24, executing process 550, one day 24 hours is divided into 24 blocks, each block is 1 hour, such as 00:00:00 to 00:59 :59, 23:00:00 to 23:59:59 is the last block, and then take out all historical performance data of the block where the time is predicted within e days, and group these vectors, the grouping method is k-means, Obtain k group, in which the similarity calculation method can adopt Euclidean Distance, after grouping is completed, before taking

The center point of the group is used to calculate the average as the predictive performance. For example, the current time is 16:50, the predicted time point is 17:00, and the performance information is collected every 10 minutes. The central processor performance information of the past month has a total of 180, and the CPU usage is 0%~10% 10 strokes, 10%~20% 15 strokes, 20%~30% 20 strokes, 30%~40% 65 strokes, 40%~50% 30 strokes, 50%~60% 15 strokes, 60% ~70% 10 strokes, 70%~80% 5 strokes, 80%~90% 5 strokes, 90%~100% 5 strokes, 100% 0 strokes, complete historical performance, please refer to the appendix, using k-means for these historical performance Do grouping, k=5, as shown in Table 3 below and Figure 8:

Among them, group 2, group 1, and group 5 are the size

Group, the average of the three groups is counted, so v _average = 42.7, 42.7 is the CPU processor usage prediction value; step S560 determines whether the prediction result exceeds T1 or the current monitoring item usage rate is lower than T2, and if it exceeds, the automatic execution is performed. Expansion module.

In summary, the present invention has achieved the desired effect under the prior art, and is not familiar with the skill of the art, and its progress and practicality have been consistent with patents. Apply for the requirements, and file a patent application in accordance with the law, and ask your bureau to approve the application for the invention patent, in order to encourage creation, to the sense of virtue.

The above is intended to be illustrative only and not limiting. Any other equivalent modifications or alterations of the present invention are intended to be included in the scope of the appended claims.

200‧‧‧Resource Usage Analysis and Prediction System for Cloud Environment

210‧‧‧Setting module

220‧‧‧Efficient collection module

230‧‧‧Performance Prediction Module

240‧‧‧Dynamic Resource Adjustment Module

Claims (5)

A cloud environment resource utilization analysis and prediction system includes: a setting module, wherein the setting module provides an input interface, and uses the input interface to set a setting parameter, wherein the setting parameter is a prediction time and a short term Analysis of historical performance quantity, number of time blocks, a long-term analysis historical performance quantity, a monitoring interval, a monitoring resource item, an automatic expansion threshold and an automatic reduction threshold; a performance collection module, the performance collection module Receiving the setting parameter of the setting module, and periodically acquiring and storing the data of the monitoring resource item according to the monitoring interval and the monitoring resource item in the setting parameter; a performance prediction module, the performance prediction module Performing a short-term performance analysis prediction and a long-term performance analysis prediction according to the data of the monitoring resource item stored in the performance collection module; and a dynamic resource adjustment module, the dynamic resource adjustment module is based on the performance prediction module The short-term performance analysis prediction and the long-term performance analysis predictive resources automatically expand resources Automatic reduction of resources. For example, the resource usage analysis and prediction system of the cloud environment described in claim 1 further includes a short-term performance analysis prediction module and a long-term performance analysis prediction module, wherein the short-term performance analysis prediction module The short-term performance analysis is used to predict the rate of change of the performance data in the prediction time to estimate the performance of the prediction time, and the long-term performance analysis prediction module uses the long-term performance analysis to predict the effectiveness of the clustering algorithm in a specific interval. Data, taking out a plurality of performance data representing the specific interval to estimate the predicted value of future resources. For example, the resource usage analysis and prediction system of the cloud environment described in claim 1 further includes a judgment module, an automatic expansion module, and an automatic reduction. The module, wherein the judging module automatically determines whether to dynamically adjust resources according to the performance prediction of the setting module and the performance prediction module, and commands the auto expansion module to automatically expand resources and automatically reduce the auto-reduction module. Reduce resources. A resource usage analysis and prediction method for a cloud environment, the method comprising: setting a parameter for predicting a prediction target through one input interface of a setting module; and collecting, by the performance collection module, the performance data of the prediction target according to the setting parameter; A performance prediction module estimates performance by predicting the rate of change of the performance data by a short-term performance analysis prediction module based on the set parameter and the performance data of the prediction target, and grouping by a long-term performance analysis prediction module by a clustering algorithm The performance data in a specific interval, the plurality of performance data representing the specific interval are extracted to estimate the predicted value of the future resource; and a dynamic resource adjustment module is based on the setting module and the performance through a determining module The performance prediction of the prediction module automatically determines whether the resource is dynamically adjusted and commands an automatic expansion module to automatically expand the resource and an automatic reduction module to automatically reduce the resource by itself. The resource usage analysis and prediction method of the cloud environment as described in claim 4, wherein the setting parameter is a prediction time, a short-term analysis historical performance quantity, a time block quantity, a long-term analysis historical performance quantity, and a Monitoring interval, a monitoring resource item, an automatic expansion threshold, and an automatic reduction threshold.