JP2018136681A - Performance management program, performance management method, and management device - Google Patents

Abstract

The present invention makes it possible to specify the processing of a performance deterioration factor.
A management apparatus determines whether performance information indicating performance of a service provided by linking a plurality of processes satisfies a performance requirement indicating performance required for the service. When the performance information does not satisfy the performance requirement, the management apparatus 10 acquires first state information indicating the operation state of each of the plurality of processes in the latest predetermined period. Further, the management device 10 satisfies the performance requirement based on the second state information 11a indicating the operation state of each of the plurality of processes when the performance of the service 1 satisfies the performance requirement, and the first state information. The difference in operation state between when the condition is satisfied and when the condition is not satisfied is calculated for each of the plurality of processes. And the management apparatus 10 determines the process which is the performance deterioration factor of the service 1 based on the difference of the operation state of each of the plurality of processes.
[Selection] Figure 1

Description

  The present invention relates to a performance management program, a performance management method, and a management apparatus.

  Cloud computing technology makes it easy to provide users with the amount of computer resources they want through the network. Among cloud computing, for example, there is PaaS (Platform as a Service) that provides a user with a platform usage environment for operating application software (hereinafter referred to as an application) via a network.

  A service using PaaS can be constructed based on a technical idea called a micro service architecture, for example. In the micro service architecture, software providing one service is created by dividing it into a plurality of small applications called components. By combining a plurality of components to provide one service, the processing capacity can be increased on a component basis. As a result, when the processing load of a certain component becomes excessive, the processing capacity of the component may be increased, and other components need not be changed.

  The execution unit of a component is called a container. When increasing the processing capacity of a component, the manager increases (scales out) the number of containers for the component to be increased, for example. By adjusting the service performance by increasing or decreasing the number of containers, system resources can be used efficiently. A PaaS system using such a container is called a Container-based PaaS Platform.

  As a technology related to efficient use of resources, for example, there is a resource management system that can improve the use efficiency of resources in response to a change in situation. In addition, as a technology related to component management, for example, there is a technology that performs monitoring based on the monitoring of the component and processing based on the monitoring result without impairing the productivity of the component of the application program.

International Publication No. 2015/049789 JP 2009-116618 A

  The administrator of the cloud computing system appropriately adjusts the performance of the component that implements the service so that the quality of the service can be maintained. For example, the administrator determines the maximum value of the latency when providing a service as a performance requirement, and if the latency of the service exceeds the maximum value, the administrator increases the processing capacity to execute the component used to provide the service Will be.

  However, just because the service latency exceeds the maximum value, it is not known which component causes the performance deterioration among a plurality of components used in the service that does not satisfy the performance requirement. In particular, in PaaS, a PaaS user creates a component, and the system administrator cannot know the specific processing contents of the component. Therefore, it is difficult for the system administrator to accurately identify the component that causes the performance deterioration.

  In addition, the problem that it is difficult to identify the process causing performance degradation is not limited to services created according to the micro service architecture, but the performance of services provided by coordinating multiple processes is adjusted. This is a problem that occurs in the same way.

  In one aspect, the purpose of this case is to make it possible to specify the processing of the performance deterioration factor.

In one proposal, a performance management program that causes a computer to execute the following processing is provided.
Based on the performance management program, the computer acquires performance information indicating the performance of a service provided by linking a plurality of processes. Next, the computer determines whether or not the performance information satisfies a performance requirement indicating the performance required for the service. Next, when the performance information does not satisfy the performance requirements, the computer acquires first state information indicating the operation states of the plurality of processes in the most recent predetermined period. Next, the computer, when the performance requirement is satisfied, based on the second state information indicating the operation state of each of the plurality of processes when the performance of the service satisfies the performance requirement, and the first state information, The difference in operating state from when it is not satisfied is calculated for each of a plurality of processes. Then, the computer determines the process that causes the performance deterioration of the service based on the difference between the operation states of the plurality of processes.

  According to one aspect, it is possible to specify the processing of the performance deterioration factor.

It is a figure which shows the structural example of the system which concerns on 1st Embodiment. It is a figure which shows the system configuration example of 2nd Embodiment. It is a figure which shows one structural example of the hardware of the management server used for this Embodiment. It is a figure which shows the concept of a micro service architecture. It is a block diagram which shows the function which a gateway and a management server have for performance adjustment. It is a figure which shows an example of the information which a latency memory | storage part memorize | stores. It is a figure which shows an example of the information which a service information storage part memorize | stores. It is a figure which shows an example of the information which a metric information storage part memorize | stores. It is a figure which shows an example of the information which a behavior storage part at normal time memorize | stores. It is a figure which shows an example of the information which a resource information storage part memorize | stores. It is a block diagram which shows the function of a performance adjustment engine. It is a figure which shows an example of the determination process of a performance requirement. It is a figure which shows the example of calculation of the behavior of a container. It is a figure which shows the example of calculation of the behavior of a server. It is a figure which shows the example of the weighting to a percentile value. It is a figure which shows the example of calculation of a factor degree. It is a figure which shows the example of an estimation of a factor component. It is a figure which shows the example of determination of a server factor degree code | symbol. It is a figure which shows the example of arrangement | positioning of a container. It is a figure which shows an example of a performance adjustment result. It is a flowchart which shows an example of the procedure of a performance adjustment process. It is the first half of the flowchart which shows an example of the procedure of the performance adjustment process in 3rd Embodiment. It is a flowchart which shows an example of the procedure of a scale-in process. It is the latter half of the flowchart which shows an example of the procedure of the performance adjustment process in 3rd Embodiment.

Hereinafter, the present embodiment will be described with reference to the drawings. Each embodiment can be implemented by combining a plurality of embodiments within a consistent range.
[First Embodiment]
First, the first embodiment will be described.

  FIG. 1 is a diagram illustrating a configuration example of a system according to the first embodiment. A service 1 provided by operating a plurality of processes (“process a”, “process b”, “process c”) in cooperation with each other is implemented in a plurality of servers 2 to 4. For example, “Processing a” is executed on the server 2, “Processing c” is executed on the server 3, and “Processing b” is executed on the server 4.

  For example, a service 1 request from the terminal device 5 is input to the server 2. Then, the server 2 executes “Processing a”. The server 2 transmits a processing request for “processing b” to the server 4 in the course of executing “processing a”. Then, the server 4 executes “processing b”. The server 4 transmits a processing request for “processing c” to the server 3 in the course of executing “processing b”. Then, the server 3 executes “processing c”. The server 3 transmits the processing result of “processing c” to the server 4. The server 4 executes the process “process b” using the process result of “process c”, and transmits the process result of “process b” to the server 2. The server 2 executes the process “process a” using the process result of “process b”, and transmits the process result of “process a” to the terminal apparatus 5 as a response to the request from the terminal apparatus 5.

  The management device 10 manages the service 1 provided by the servers 2 to 4. For example, the management apparatus 10 adjusts the performance of the service 1. Specifically, when the performance of the service 1 deteriorates, the management apparatus 10 specifies a process that causes the performance deterioration of the service 1. And the management apparatus 10 controls the process performed by the servers 2-4 so that performance degradation is eliminated.

  Here, the difficulty of specifying the process that causes the performance deterioration of the service 1 will be described. As shown in FIG. 1, in the case of a service 1 provided by linking a plurality of processes, just because the performance of the service 1 has deteriorated, it cannot be understood which component has a factor of performance deterioration.

  Therefore, it is conceivable to define performance requirements for each component. However, it is difficult to accurately determine what kind of performance requirement is defined for each component and whether the performance requirement of the service can be satisfied. For example, in order to keep the service latency within 100 milliseconds, it is necessary to accurately determine the appropriate value of CPU (Central Processing Unit) usage rate, memory usage rate, disk I / O rate, etc. for each component. It is difficult to decide. In addition, in the case of a component created by a service user, the administrator cannot know the specific processing contents of the component. It is difficult to determine the performance requirements of a component without knowing the processing content of the component.

  Therefore, the management device 10 implements a performance management method for accurately identifying the process that causes the performance deterioration based on the operation state of the process in each of the servers 2 to 4. For this purpose, the management apparatus 10 includes a storage unit 11 and a processing unit 12 as described below. The storage unit 11 is, for example, a memory or a storage device included in the management device 10. The processing unit 12 is, for example, one or more processors included in the management device 10. The processing executed by the processing unit 12 can be realized, for example, by causing a processor to execute a performance management program in which the processing procedure is described.

  The storage unit 11 indicates the operation state of each of the plurality of processes when the performance of the service 1 provided by linking the plurality of processes satisfies the performance requirement indicating the performance required for the service 1. The state information 11a is stored. The second state information 11a is a plurality of types of information such as a CPU usage rate of each process and a memory I / O rate at the time of executing each process. Information indicating such an operating state is called a metric. Note that the storage unit 11 may store values resulting from the statistical processing of various metrics as the second state information 11a. For example, the percentile value of the CPU usage rate for each process can be used as the second state information 11a.

  The processing unit 12 acquires performance information indicating the performance of the service 1. For example, the processing unit 12 monitors communication between the terminal device 5 and the server 2 and acquires a time (latency) from a request to a response. For example, the processing unit 12 calculates an index value of performance such as Addex based on the latency for a plurality of requests. The index will be described later.

  The processing unit 12 determines whether the acquired performance information satisfies the performance requirement. For example, it is assumed that the performance requirement specifies that the index is 0.8 or more. In this case, the processing unit 12 determines whether or not the Index value calculated based on the acquired performance information is 0.8 or more.

  When the performance information does not satisfy the performance requirement, the processing unit 12 acquires, from the servers 2 to 4, first state information indicating the operation states of the plurality of processes in the latest predetermined period. For example, the processing unit 12 acquires metric values such as the CPU usage rate of each process and the memory I / O rate when each process is executed.

  Based on the acquired first state information and second state information 11a, the processing unit 12 calculates a difference in operating state between when the performance requirement is satisfied and when it is not satisfied for each of the plurality of processes. To do. For example, the processing unit 12 calculates a representative value (for example, a percentile value) of a metric value for the most recent predetermined period based on the acquired first state information. Then, the processing unit 12 calculates a difference between the representative value calculated from the first state information and the representative value calculated from the second state information 11a.

  And the process part 12 determines the process used as the performance deterioration factor of the service 1 based on the difference of the operation state of each of several process. For example, the processing unit 12 determines a process with the largest difference in operation state as a process of performance deterioration factor.

  The processing unit 12 further determines a coping method for the performance deterioration based on the difference in the operation state of the factor processing determined to be the performance deteriorating factor, and performs coping with the determined coping method. For example, the processing unit 12 scales out the factor processing.

  In this way, it is possible to determine whether the process used to provide the service 1 is a factor that deteriorates performance. As a result, it is possible to quickly cope with the performance deterioration of the service 1. Further, it is not necessary to set performance requirements for each metric for each process, and the management burden on the system is reduced.

  In addition, the process part 12 can also improve the precision of the 2nd status information 11a by updating the 2nd status information 11a suitably. For example, when the performance information of the service 1 satisfies the performance requirement, the processing unit 12 acquires the third state information indicating the operation state of each of the plurality of processes in the latest predetermined period. Then, the processing unit 12 updates the second state information 11a based on the acquired third state information. For example, based on the third state information of a plurality of periods, the processing unit 12 reflects the operation state indicated by the third state information in a period closer to the present in the updated second state information 11a. As described above, the second state information 11a is updated with the latest performance information, and the second state information 11a is updated by increasing the weight of the new update information, thereby reflecting the recent operation status of the system. High second state information 11a can be generated.

  For example, when the performance of the service 1 satisfies the performance requirement, the storage unit 11 stores, as the second state information 11a, the second resource information indicating the time series change of the operation status of the resources used by each of the plurality of processes. A second representative value that is a predetermined representative value may be stored. In this case, the processing unit 12 acquires, as the first state information, first resource information indicating a time-series change in the operating status of resources used by each of the plurality of processes in the most recent predetermined period, and the first resource information Is determined as a first representative value. Then, the processing unit 12 calculates the difference between the first representative value and the second representative value for each of the plurality of processes, and determines that the process with the largest difference is a factor process that is a factor of performance deterioration. Thus, by representing the operation status of the resource by the representative value, the difference in the operation state can be easily quantified. As a result, it is possible to easily identify a process whose operation state has largely changed before and after the performance deterioration of the resource 1.

  Note that when the values of multiple types of metrics (CPU usage rate, memory I / O rate, etc.) are acquired as the first status information and the second status information 11a, the processing unit 12 displays the representative value for each metric type. Calculate the difference. The processing unit 12 can also calculate a plurality of types of representative values (for example, 50th percentile, 90th percentile, 99th percentile, etc.) for one type of metric. In this case, the processing unit 12 calculates, for each process, the difference between the same kind of representative values of the same kind of metrics of the first state information and the second state information 11a. Then, the processing unit 12 sums up the differences (for example, absolute values) between the representative values for each metric type (for example, CPU usage rate) of each process, and sets the difference in the operation state regarding the corresponding metric type of the corresponding process. Further, the processing unit 12 determines the difference between the representative values that increase when the performance deteriorates (referred to as “positive factor” in the second embodiment) and the difference between the representative values that decrease when the performance deteriorates (second In this embodiment, it is also possible to calculate “negative factor degree” individually.

  For example, the processing unit 12 can perform scale-out of factor processing as a countermeasure against the performance deterioration of the service 1. Further, when the performance of the service 1 is deteriorated due to the influence of processes other than the factor process in the server that is currently executing the factor process, the processing unit 12 can change the server that performs the factor process. For example, the processing unit 12 represents an operation state in which the second state information 11a of the factor processing (state information at the time of performance deterioration) has a larger load than the first state information of the factor processing (state information at normal time). If there is, scale out the factor processing. Further, the processing unit 12 changes the server that executes the factor process when the first state information of the factor process represents an operation state with a larger load than the second state information 11a of the factor process. Thereby, execution of useless scale-out can be suppressed.

  Further, the processing unit 12 may perform the scale-in when it is confirmed that the scale-out is excessive after the change of the factor process and the scale-out are performed at the same time. For example, the processing unit 12 first stops the execution of the factor process on the first server that is currently executing the factor process, and causes the factor server to execute the factor process on each of a plurality of second servers different from the first server. Then, the processing unit 12 stops the execution of the factor processing in a part of the plurality of second servers when the processing load required for the plurality of second servers to execute the factor processing after the execution of the countermeasure is equal to or less than a predetermined value. Thereby, the performance deterioration state of the service 1 can be quickly resolved, and wasteful resource consumption can be suppressed.

[Second Embodiment]
Next, a second embodiment will be described. The second embodiment is a computer that can accurately determine a component having an excessive load when the service latency exceeds the maximum value when performing operation management of PaaS constructed based on the micro service architecture. System.

  FIG. 2 is a diagram illustrating a system configuration example according to the second embodiment. A plurality of terminal devices 31, 32,... Are connected to the cloud computing system 40 via the network 20. The cloud computing system 40 provides a service by PaaS to a plurality of terminal devices 31, 32,.

  The cloud computing system 40 includes a gateway 41, a management server 100, and a plurality of servers 42 to 44. The gateway 41 is connected to the network 20 and accepts requests from a plurality of terminal devices 31, 32,. The management server 100 is connected to the gateway 41 and the plurality of servers 42 to 44 and manages the plurality of servers 42 to 44. The plurality of servers 42 to 44 provide information processing services in response to requests from the plurality of terminal devices 31, 32,.

  FIG. 3 is a diagram illustrating a configuration example of hardware of the management server used in the present embodiment. The management server 100 is entirely controlled by a processor 101. A memory 102 and a plurality of peripheral devices are connected to the processor 101 via a bus 109. The processor 101 may be a multiprocessor. The processor 101 is, for example, a CPU, an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). At least a part of the functions realized by the processor 101 executing the program may be realized by an electronic circuit such as an ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device).

  The memory 102 is used as a main storage device of the management server 100. The memory 102 temporarily stores at least part of an OS (Operating System) program and application programs to be executed by the processor 101. The memory 102 stores various data necessary for processing by the processor 101. As the memory 102, for example, a volatile semiconductor storage device such as a RAM (Random Access Memory) is used.

  Peripheral devices connected to the bus 109 include a storage device 103, a graphic processing device 104, an input interface 105, an optical drive device 106, a device connection interface 107, and a network interface 108.

  The storage device 103 writes and reads data electrically or magnetically with respect to a built-in recording medium. The storage device 103 is used as an auxiliary storage device of a computer. The storage device 103 stores an OS program, application programs, and various data. For example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive) can be used as the storage device 103.

  A monitor 21 is connected to the graphic processing device 104. The graphic processing device 104 displays an image on the screen of the monitor 21 in accordance with an instruction from the processor 101. Examples of the monitor 21 include a display device using a CRT (Cathode Ray Tube) and a liquid crystal display device.

  A keyboard 22 and a mouse 23 are connected to the input interface 105. The input interface 105 transmits signals sent from the keyboard 22 and the mouse 23 to the processor 101. The mouse 23 is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball.

  The optical drive device 106 reads data recorded on the optical disc 24 using laser light or the like. The optical disc 24 is a portable recording medium on which data is recorded so that it can be read by reflection of light. The optical disc 24 includes a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable) / RW (ReWritable), and the like.

  The device connection interface 107 is a communication interface for connecting peripheral devices to the management server 100. For example, the memory device 25 and the memory reader / writer 26 can be connected to the device connection interface 107. The memory device 25 is a recording medium equipped with a communication function with the device connection interface 107. The memory reader / writer 26 is a device that writes data to the memory card 27 or reads data from the memory card 27. The memory card 27 is a card type recording medium.

  The network interface 108 is connected to the network 20. The network interface 108 transmits and receives data to and from other computers or communication devices via the network 20.

  With the hardware configuration as described above, the processing function of the management server 100 in the second embodiment can be realized. The terminal devices 31, 32,..., The gateway 41, and the servers 42 to 44 can also be realized by the same hardware as the management server 100. Further, the management apparatus 10 shown in the first embodiment can also be realized by the same hardware as the management server 100 shown in FIG.

  The management server 100 implements the processing functions of the second embodiment by executing a program recorded on a computer-readable recording medium, for example. The program describing the processing contents to be executed by the management server 100 can be recorded in various recording media. For example, a program to be executed by the management server 100 can be stored in the storage device 103. The processor 101 loads at least a part of the program in the storage apparatus 103 into the memory 102 and executes the program. A program to be executed by the management server 100 can also be recorded on a portable recording medium such as the optical disc 24, the memory device 25, and the memory card 27. The program stored in the portable recording medium becomes executable after being installed in the storage apparatus 103 under the control of the processor 101, for example. The processor 101 can also read and execute a program directly from a portable recording medium.

In the second embodiment, software providing a service is installed in the servers 42 to 44 based on the micro service architecture.
FIG. 4 is a diagram illustrating the concept of the micro service architecture. The service 50 provided to the user is realized using a plurality of components 51 to 53. For example, the component 51 is software that executes processing of the presentation layer, the component 52 is software that executes processing of the logic layer, and the component 53 is software that executes processing of the data layer.

The components 51 to 53 are executed by any one or more of the plurality of servers 42 to 44. A processing function constructed on the servers 42 to 44 by executing the components 51 to 53 is a container. In the second embodiment, the container is represented as “C _xy ”. The subscript “x” is an identification number (component number) of a component including the container. The subscript “y” is an identification number (container number) of a container in a component including the container.

  As described above, in the micro service architecture, software for providing one service 50 is created by being divided into a plurality of small components 51 to 53. Each component 51-53 is loosely coupled. The loose coupling means that the components 51 to 53 are relatively loosely coupled and have a strong independence. Since the coupling of the components 51 to 53 is sparse, there is an advantage that a change of other components due to addition of a new component or expansion of some components can be reduced.

  The service components 51 to 53 created according to the micro service architecture are executed by the container. The components 51 to 53 and the container have a one-to-many relationship.

  The performance requirement required for the service 50 provided to the user can be expressed using, for example, latency. Therefore, the system administrator prepares the components 51 to 53 having the processing capability so that the latency required for the service 50 can be obtained. The processing capacity of the components 51 to 53 can be adjusted by increasing or decreasing the number of containers that execute the components 51 to 53.

  Here, it is easy for the administrator to specify performance requirements required for the service 50. On the other hand, it is difficult for the administrator to determine how much resources should be allocated to each component so as to satisfy the latency required for the service 50. Therefore, in the second embodiment, the management server 100 detects a component having insufficient performance, and adds a container for executing the component, thereby adding a resource to the component that satisfies the performance requirement for the service 50. Realize the allocation.

  FIG. 5 is a block diagram illustrating functions of the gateway and the management server for performance adjustment. The gateway 41 includes a latency measurement unit 41a and a latency storage unit 41b. The latency measuring unit 41a measures the time from when a request is received from the terminal device 31, 32,... Until a response corresponding to the request is transmitted to the terminal device 31, 32,. The latency measuring unit 41a stores the measured time in the latency storage unit 41b as the latency for the service according to the request. The latency storage unit 41b stores the latency measured by the latency measurement unit 41a.

  The management server 100 includes a service information storage unit 110, a metric information storage unit 120, a normal behavior storage unit 130, a resource information storage unit 140, and a performance adjustment engine 150. The service information storage unit 110 stores information related to the service to be provided. The metric information storage unit 120 stores information (metric) on the operating status of resources by the servers 42 to 44 and the containers. The normal behavior storage unit 130 stores information indicating the behavior during normal operation of each of the plurality of containers and the plurality of servers. The resource information storage unit 140 stores information regarding resources used by the servers 42 to 44. The performance adjustment engine 150 uses the information stored in the service information storage unit 110, the metric information storage unit 120, the normal behavior storage unit 130, and the resource information storage unit 140 to perform performance adjustment in units of components.

  In the following description, mounting a container that executes component processing on a server is referred to as container arrangement. Specifically, the container arrangement is a process of installing a program for executing a component in a server and starting a process for executing the component process based on the program. Also, when a container is mounted on a server, it is said that the container is placed on that server.

In the example of FIG. 5, each of the servers 42 to 44 has a plurality of containers having different components. For example, containers C ₁₁ , C ₂₂ and C ₃₁ are arranged in the server 42.
Hereinafter, the information stored in the service information storage unit 110, the metric information storage unit 120, the normal behavior storage unit 130, and the resource information storage unit 140 will be described in detail with reference to FIGS.

  FIG. 6 is a diagram illustrating an example of information stored in the latency storage unit. The latency storage unit 41b stores, for example, a latency management table 41c. The latency management table 41c has columns of time stamp, request ID, service name, and latency.

  In the time stamp column, the date and time when the latency is measured is set. In the request ID column, identification information (request ID) of a request for measuring latency is set. In the service name column, the name of the service (service name) corresponding to the request whose latency has been measured is set. The measured latency is set in the latency column.

  FIG. 7 is a diagram illustrating an example of information stored in the service information storage unit. The service information storage unit 110 stores a service management table 111, for example. The service management table 111 includes columns for a service name, an index (Application performance index), a satisfied time, and a component name. In the service name column, the name of the service provided (service name) is set. In the column of “Adex”, performance requirements required for the corresponding service are set by the “Adex”. The index is an index indicating the degree of user satisfaction with respect to latency. In the field of Satified Time, the maximum latency value (T) that the user who uses the corresponding service is considered to be satisfied is set. In the component name column, the name of the component used for providing the service is set.

Here, the Index will be described in detail. Addex is an index standardized by “The Alliance” and is calculated by the following equation.
Addex = ((satisfied counts) + (tolerating counts) / 2) / (total counts)
“Satisfied counts” is the number of requests whose latency is T or less. In other words, “satisfied counts” is the number of requests for which a latency satisfying the user is obtained.

  “Tollating counts” is the number of requests with a latency of T or more and 4 × T or less. In other words, “tolerating counts” is the number of requests for which an acceptable latency is obtained, although the latency is not satisfactory for the user.

  The number of requests with a latency greater than 4 × T is called “frustrated”. This “frustrated” is the number of requests that resulted in a latency that the user felt dissatisfied with.

  In the second embodiment, if the value of the Index calculated based on the service latency is equal to or higher than the Index value set as the performance requirement, it is determined that the performance requirement is satisfied. Conversely, if the value of the Index calculated based on the service latency is less than the Index value set as the performance requirement, it is determined that the performance requirement is not satisfied.

  FIG. 8 is a diagram illustrating an example of information stored in the metric information storage unit. The metric information storage unit 120 stores a metric management table 121, for example. The metric management table 121 has columns of time stamp, server / container name, metric type, and value. The date and time when the metric value is measured is set in the time stamp column. In the server / container name column, the name of the server or container that has measured the metric value is set. In the metric type column, the measured metric type (metric type) is set. In the value column, the value of the measured metric is set.

  FIG. 9 is a diagram illustrating an example of information stored in the normal behavior storage unit. The normal behavior storage unit 130 stores, for example, a plurality of container behavior management tables 131a, 131b,... For each behavior measurement period, and a plurality of server behavior management tables 132a, 132b,. doing.

  The plurality of container behavior management tables 131a, 131b,... Are provided in association with the measurement cycle of the container behavior. The plurality of container behavior management tables 131a, 131b,... Have columns for containers, metric types, percentile types, percentile values, and weighted percentile values. In the container column, the name (container name) of the container whose behavior is to be measured is set. In the metric type column, the type of metric whose behavior is measured is set. In the percentile type field, the percentile type to be obtained for the metric value is set. For example, the 50th percentile, 90th percentile, 99th percentile, etc. are set as the type of percentile. In the percentile value column, the percentile value indicated by the type of the percentile for the corresponding metric is set. In the column of weighted percentile values, weighted percentile values for each metric of the container based on metric values for the past several cycles are set. Details of the weighted percentile value will be described later (see FIG. 15).

  The plurality of server behavior management tables 132a, 132b,... Are provided in association with the server behavior measurement cycle. The plurality of server behavior management tables 132a, 132b,... Have columns of server, metric type, percentile type, percentile value, and weighted percentile value. In the server column, the name (server name) of the server whose behavior is to be measured is set. In the metric type column, the type of metric whose behavior is measured is set. In the percentile type field, the percentile type to be obtained for the metric value is set. For example, the 50th percentile, 90th percentile, 99th percentile, etc. are set as the type of percentile. In the percentile value column, the percentile value indicated by the percentile type for the corresponding server is set. In the column of weighted percentile values, weighted percentile values for each metric of the server based on metric values for the past several cycles are set.

  The percentile is a kind of representative value of statistics. When a plurality of data are arranged in order of size, the proportion of data smaller than the value x (x is a real number) is less than p% (p is a real number of 0 to 100), and the proportion of data larger than that is “100−p The value x that is “%” is the p percentile. The p percentile is also called the pth percentile.

  FIG. 10 is a diagram illustrating an example of information stored in the resource information storage unit. The resource information storage unit 140 stores, for example, a container arrangement management table 141, a server resource management table 142, and a container resource management table 143.

  The container placement management table 141 is a data table that manages the placement status of containers on the servers 42 to 44. The container arrangement management table 141 has columns for server name and container name. The name of the server (server name) on which the container is mounted is set in the server name column. In the container name column, the name of the container (container name) installed in the corresponding server is set.

  The server resource management table 142 is a data table for managing the amount of free resources of the servers 42 to 44. The server resource management table 142 includes columns for server name and remaining resource amount. In the server name column, the name of the server (server name) used for providing the service is set. In the remaining resource amount column, the free amount of resources of the corresponding server (remaining resource amount) is set for each resource type. In the example of FIG. 9, the remaining resource amounts of the CPU, memory, and network are set.

  The container resource management table 143 is a data table for managing the amount of resources used by each component container. The container resource management table 143 has columns of components and container usage resource amounts. In the component column, the name of the component (component name) used for providing the service is set. The amount of resource used by the container of the corresponding component is set for each resource type in the column for the amount of resource used by the container. In the example of FIG. 9, the used resource amount of the container for the CPU, memory, and network is set.

Next, the performance adjustment engine 150 will be described in detail.
FIG. 11 is a block diagram illustrating functions of the performance adjustment engine. The performance adjustment engine 150 includes a service management unit 151, a metric information collection unit 152, a latency inspection unit 153, a behavior calculation unit 154, an abnormality factor estimation unit 155, and a container arrangement control unit 156.

  The service management unit 151 manages service configuration and performance requirements. The metric information collection unit 152 periodically collects metric values from the servers 42 to 44 and stores them in the metric information storage unit 120. The latency checking unit 153 checks whether the service latency satisfies the performance requirement. The behavior calculation unit 154 calculates the behavior of the container and the server at normal time and abnormal time. The behavior calculation unit 154 stores the normal behavior in the normal behavior storage unit 130. The abnormality factor estimation unit 155 estimates a component (factor component) that is an abnormality factor of a service whose latency does not satisfy the performance requirement. The container arrangement control unit 156 performs scale-out of the factor component or changes the arrangement of the container that executes the factor component.

  In addition, the line which connects between each element shown in FIG. 11 shows a part of communication path, and communication paths other than the illustrated communication path can also be set. Moreover, the function of each element shown in FIG. 11 can be realized, for example, by causing a computer to execute a program module corresponding to the element.

Next, the determination process in the performance adjustment engine 150 for determining whether or not each service satisfies the performance requirement will be described.
FIG. 12 is a diagram illustrating an example of a performance requirement determination process. The service management unit 151 registers the Index value in the service information storage unit 110 as the performance requirement of the service 50 according to the input of the administrator. For example, the service management unit 151 receives an input of an Index value and Satisfied Time (T) from the administrator. Then, the service management unit 151 stores the input Index value and Satisfied Time (T) in the service management table 111 in association with the service name of the service 50.

  The latency checking unit 153 periodically collects latencies relating to requests to the service 50 within the latest predetermined period from the gateway 41. The service latency is the difference between the reception time of the request issued from the terminal device 31 at the gateway 41 and the transmission time of the response from the gateway 41 to the terminal device 31. The latency inspecting unit 153 calculates an Index value for a predetermined period based on the acquired latency. The latency checking unit 153 determines that the performance requirement is satisfied if the calculated Index value is equal to or greater than the Index value specified as the performance requirement. The latency checking unit 153 determines that the performance requirement is not satisfied when the calculated Index value is less than the Index value specified as the performance requirement.

  Next, metric information of the container and the server is collected by the metric information collection unit 152 and stored in the metric information storage unit 120. The collected metric information includes, for example, a CPU usage rate, a memory I / O rate and the number of page faults, a disk (file system) I / O rate, a network transmission / reception rate, and the like. Based on the collected metric information, the behavior calculation unit 154 calculates the behavior of the container and the server in the latest predetermined period.

FIG. 13 is a diagram illustrating a calculation example of the behavior of the container. In the example of FIG. 13, it is assumed to calculate the behavior of the container C _11. The behavior calculation unit 154 extracts a record whose container name is “C ₁₁ ” from the metric information storage unit 120. Next, the behavior calculation unit 154 classifies the extracted records by metric type. Next, the behavior calculation unit 154 normalizes the values (metric values) set in the records of the same metric type to be 0 to 100, and generates a frequency distribution. For example, the behavior calculation unit 154 normalizes so that the theoretical maximum value of each metric value becomes “100”. Then, the behavior calculation unit 154 calculates a 50th percentile value, a 90th percentile value, and a 99th percentile value for each metric type based on the frequency distribution.

  The behavior calculation unit 154 calculates the behavior of all containers that execute the components of the service 50. When the latency checking unit 153 determines that the performance requirement of the service 50 is satisfied, the behavior calculation unit 154 creates a container behavior management table 131a of the latest cycle, and stores the container behavior management table 131a. It is stored in the normal behavior storage unit 130.

  FIG. 14 is a diagram illustrating a calculation example of the behavior of the server. In the example of FIG. 14, it is assumed that the behavior of the server 42 with the server name “server 1” is calculated. The behavior calculation unit 154 extracts a record whose server name is “server 1” from the metric information storage unit 120. Next, the behavior calculation unit 154 classifies the extracted records by metric type. Next, the behavior calculation unit 154 normalizes the values (metric values) set in the records of the same metric type to be 0 to 100, and generates a frequency distribution. Then, the behavior calculation unit 154 calculates a 50th percentile value, a 90th percentile value, and a 99th percentile value for each metric type based on the frequency distribution.

  The behavior calculation unit 154 calculates the behavior of all the servers 42 to 44. When the latency checking unit 153 determines that the performance requirement of the service 50 is satisfied, the behavior calculation unit 154 creates the server behavior management table 132a of the most recent cycle and stores the server behavior management table 132a. It is stored in the normal behavior storage unit 130.

  When the latency checking unit 153 determines that the performance requirement of the service 50 is not satisfied, the behavior calculation unit 154 uses the calculated percentile value between the container and the server as information indicating the behavior at the time of abnormality, and estimates the cause of the abnormality To the unit 155. Then, the abnormality factor estimation unit 155 compares the behavior at the time of abnormality with the behavior at the time of normality, and estimates a component that causes a decrease in service latency.

  For example, the abnormality factor estimation unit 155 acquires a percentile value for each metric of the container for n cycles (n is an integer equal to or greater than 1) from the newest behavior storage unit 130. Then, the abnormality factor estimation unit 155 determines the normal behavior of each metric based on the acquired percentile value. At this time, the abnormality factor estimation unit 155 weights the percentile value according to the age of the cycle from which the percentile value is acquired in order to consider that the behavior with the cycle closer to the present is closer to the future behavior.

  FIG. 15 is a diagram illustrating an example of weighting the percentile value. In the example illustrated in FIG. 15, it is assumed that the percentile values at the normal time for three periods of the period t to t + 2 are acquired. At this time, the abnormality factor estimation unit 155 sets the weight of the percentile value of the latest period t + 2 to “3”. In addition, the abnormality factor estimation unit 155 sets the weight of the percentile value of the immediately preceding cycle t + 1 to “2”. Furthermore, the abnormality factor estimation unit 155 sets the weight of the percentile value of the previous period t to “2”.

  As described above, the abnormality factor estimation unit 155 increases the weight as the percentile value of the cycle closer to the current time, and calculates the percentile value (weighted percentile value) of the period of n cycles for each metric. For example, the weighted percentile value is calculated as follows.

It is assumed that the following data is obtained as the percentile value at the normal time. S1 is a set of data of the latest cycle. S2 is a data set of the cycle immediately before S1. S3 is a data set of the cycle immediately before S2.
S1: {1, 2}
S2: {3, 4}
S3: {5, 6}
In this example, the data value is simplified to make the weighting process easier to understand. When obtaining weighted percentile values for S1, S2, and S3, the number of each normal data is increased by the weight. For example, the weights for the sets S1, S2, and S3 are “3”, “2”, and “1”, respectively. In this case, the sets S1, S2, and S3 are replaced with the following sets.
S1 ′ = S1 × 3: {1, 1, 1, 2, 2, 2}
S2 ′ = S2 × 2: {3, 3, 4, 4}
S3 ′ = S3 × 1: {5, 6}
The set S1 ′ is a triple of the set S1. That is, the set S1 ′ is a set of the same three sets as the set S1. The set S2 ′ is a double of the set S2. That is, a set S2 ′ is a set of the same two sets as the set S2. The set S3 ′ is the same as the set S3. The abnormality factor estimation unit 155 combines these sets S1 ′ and S2′S3 ′ into one set, and sorts the data in ascending order. That is, the abnormality factor estimation unit 155 generates, for each set for each period, the same set as the set for the number of weights, collects the generated sets into one, and sorts the data in ascending order. As a result of the sorting, the following set S is obtained.
S =: {1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 5, 6}
The abnormality factor estimation unit 155 sets the percentile value obtained based on the set S as a weighted percentile value. Then, the 50th percentile becomes “2”. The 90th percentile is “4”.

  The abnormality factor estimation unit 155 compares the weighted percentile value at the normal time and the latest percentile value indicating the behavior at the time of the abnormality for each metric type, and obtains a factor degree related to the metric type. The abnormality factor estimation unit 155 obtains, for example, a positive factor degree and a negative factor degree as factor degrees.

  FIG. 16 is a diagram illustrating a calculation example of the factor degree. In the example of FIG. 16, in the weighted percentile value indicating the normal behavior, the 50th percentile value is “15”, the 90th percentile value is “71”, and the 99th percentile value is “90”. In the latest percentile value indicating the behavior at the time of abnormality, the 50th percentile value is “6”, the 90th percentile value is “92”, and the 99th percentile value is “98”.

Here, the positive factor degree and the negative factor degree are determined as follows.
-Positive factor F ₊ = Σ (increment of P of P percentile with increasing value) × (difference of percentile value)
Negative factor F ₋ = Σ (increment of P of P percentile for which value decreases) × (difference of percentile value)
P is a numerical value indicating the type of percentile, and P = 50 for the 50th percentile. The P percentile whose value increases is a percentile type in which the percentile value at the time of abnormality is larger than the percentile value at the time of normality. The P percentile whose value decreases is a percentile type in which the normal percentile value is larger than the normal percentile value.

  The P increment of the P percentile is the amount of increase in the P value from the previous percentile type for each percentile type when the percentile types are arranged in ascending order of the P value. In the example of FIG. 16, there are 50th percentile, 90th percentile, and 99th percentile. In that case, the increment of P for the 50th percentile is “50”. The increment of P for the 90th percentile is “40” (90-50). The increment of P for the 99th percentile is “9” (99-90).

  When the service latency does not meet the performance requirement, if the load on the container or server is increased than usual, the metric value is concentrated on a high value, and the positive factor is high. Also, when the service latency does not satisfy the performance requirements, if the load on the container or server is lower than normal, the metric value is concentrated at a low value, and the negative factor is high. If the service latency meets the performance requirements but the negative factor is higher than the positive factor of the container or server, the performance is degraded due to a factor other than that of the container or server. I can judge.

In the example shown in FIG. 16, the factor degrees are as follows.
Positive factor F ₊ = (90−50) × (92−71) + (99−90) × (98−90) = 912
・ Negative factor F ₋ = 50 × (15−6) = 450
The abnormality factor estimation unit 155 performs the calculation of the factor degree for each metric type. Then, the abnormality factor estimation unit 155 estimates the component executed by the container from which the maximum factor degree is calculated as the factor component that is the cause of the abnormality.

FIG. 17 is a diagram illustrating an example of estimating the factor component. As shown in FIG. 17, the positive factor and the negative factor are calculated for each metric type for all containers. The abnormality factor estimation unit 155 extracts the maximum factor degree from the calculated factor degrees. In the example of FIG. 17, the value of the positive factors of about CPU usage of the container C ₁₁ is maximum. The abnormality factor estimation unit 155 estimates the component (component name “component 1”) executed in the container C ₁₁ which is the calculation source of the extracted factor degree as the factor component. At this time, the abnormality factor estimation unit 155 sets the metric type “CPU usage rate” corresponding to the maximum factor degree as the factor metric. Also, the abnormality factor estimation unit 155 sets the container factor degree code indicating whether the maximum factor factor is a positive factor factor or a negative factor factor to be positive.

  Furthermore, the abnormality factor estimation unit 155 acquires the server name of the server on which the container that is the source of calculation of the maximum factor degree is mounted from the container arrangement management table 141. Then, the abnormality factor estimation unit 155 sets the acquired server name as the server name of the container operation server. In the example of FIG. 17, the container operating server is “Server 1”.

  Also, the abnormality factor estimation unit 155 calculates a factor degree for each metric type for the server. Then, the abnormality factor estimation unit 155 compares the positive factor degree with the negative factor degree for each metric type of the server. If the positive factor degree is equal to or greater than the negative factor degree, the abnormality factor estimating unit 155 sets the factor degree code of the metric type to “positive”. If the positive factor degree is less than the negative factor degree, the abnormality factor estimating unit 155 sets the factor degree code of the metric type to “negative”.

Then, the abnormality factor estimation unit 155 sets the factor degree code of the factor metric of the container operation server as the server factor degree code.
FIG. 18 is a diagram illustrating a determination example of the server factor degree code. In the example of FIG. 18, since the factor degree code of the factor metric “CPU usage rate” of the container operation server “server 1” is “positive”, the server factor degree code is “positive”.

The server factor can also be calculated in the same procedure as the container, but for the server, the factor code for each metric type only needs to be known. Therefore, for example, the factor degree of the metric type may be calculated by the following formula without dividing the positive factor degree and the negative factor degree.
・ Factor factor F = Σ (increment of P of P percentile) × (difference of percentile value)
The difference between the percentile values at this time is a value obtained by subtracting the abnormal percentile value from the normal percentile value. If the factor F calculated in this way is a value of 0 or more, the factor code is “positive”. If the factor F is a negative value, the factor sign is “negative”.

  When the abnormal factor estimation unit 155 determines the factor component, factor metric, maximum factor code, and server factor degree code, the container arrangement control unit 156 adds a container or changes the container arrangement destination so as to improve the latency. Perform performance improvement processing.

  For example, when the container factor degree code is positive, the container arrangement control unit 156 determines that the resource of the factor component is insufficient, and scales out the factor component. In addition, when the factor degree of the factor component is negative and the server factor degree code is “positive”, the container arrangement control unit 156 is affected by a large resource load by components other than the factor component, and the performance of the factor component Is judged to have declined. In this case, the container arrangement control unit 156 performs container arrangement conversion. Container layout conversion is a process of changing a server that operates a container to another server.

  Note that the amount of resources used by a component container may be specified. In this case, the container placement control unit 156 sets a server that can accommodate the container as a placement destination candidate when the component is scaled out or placed. When there are a plurality of servers that are placement destination candidates, the container placement control unit 156 assumes that the placement destination candidate having the largest minimum residual resource amount of the server is placed in the placement destination, assuming that the container is placed in each placement destination candidate. decide.

  FIG. 19 is a diagram illustrating an exemplary arrangement of containers. In the example of FIG. 19, the factor component is “component 1”, and the container factor degree code is “positive”. In this case, the container arrangement control unit 156 scales out “component 1”.

  At this time, the container arrangement control unit 156 refers to the server resource management table 142 and confirms the remaining resource amount of each server. In the example of FIG. 19, the remaining resource amount of “server 1” is CPU “50”, memory “30”, and network “40”. The remaining resource amount of “Server 2” is CPU “30”, memory “50”, and network “60”.

  Further, the container arrangement control unit 156 refers to the container resource management table 143 and confirms the amount of resources used per container of the factor component. In the example of FIG. 19, the resources used in the container of “component 1” that is the factor component are the CPU “10”, the memory “20”, and the network “10”.

  Here, it is assumed that only the server 42 having the server name “server 1” and the server 43 having the server name “server 2” have the remaining resource amount sufficient to place the container of “component 1”. In this case, the server 42 and the server 43 are placement destination candidates.

  The remaining resource amount when the container is arranged in the server 42 with the server name “server 1” is the CPU “40”, the memory “10”, and the network “30”. When the container is arranged in the server 43 with the server name “server 2”, the remaining resource amounts are the CPU “20”, the memory “30”, and the network “50”. In this case, the minimum remaining resource amount of the server 42 with the server name “server 1” is “10” in the memory. On the other hand, the minimum remaining resource amount of the server 43 with the server name “server 2” is “20” of the CPU.

The container placement control unit 156 selects the server 43 with the server name “server 2” having the smallest minimum residual resource amount as the placement destination. The container placement control unit 156 then sends the server 43 a scale-out process. A container C ₁₃ for executing “component 1” is arranged.

  The container arrangement control unit 156 continues the performance adjustment until the Index value reaches the target value. Then, when the Addex value reaches the target value, the container placement control unit 156 ends the performance adjustment.

  FIG. 20 is a diagram illustrating an example of the performance adjustment result. In the example of FIG. 20, the target value of the Index value is 0.8 or more. Before the performance adjustment, the Apdex value was “0.75”, but by performing the performance adjustment, the Index value is improved to “0.83”.

Next, the procedure of the performance adjustment process will be described in detail.
FIG. 21 is a flowchart illustrating an example of the procedure of the performance adjustment process. Note that the processing shown in FIG. 21 is processing when performance adjustment is performed for one service. When performance adjustment is performed for a plurality of services, the process illustrated in FIG. 21 is performed for each of the plurality of services. In the following, the process illustrated in FIG. 21 will be described in order of step number.

  [Step S101] The performance adjustment engine 150 initializes the value of the variable R indicating the number of repetitions to “0” when, for example, an administrator inputs a service performance adjustment processing start instruction.

  [Step S102] The latency checking unit 153 acquires service information about the performance adjustment target service and the latency of the service. For example, the latency checking unit 153 acquires service information from the service information storage unit 110. The service information to be acquired includes the value of Index specified as a performance requirement, and Satisfied Time (T) used for calculating the Index. Further, the latency checking unit 153 acquires the latency of the request for the performance adjustment target service, which is measured within the latest predetermined period, from the latency storage unit 41 b of the gateway 41.

[Step S103] The latency checking unit 153 calculates the service index based on the latency of the plurality of requests.
[Step S104] The latency checking unit 153 determines whether or not the value of the Index calculated in Step S103 satisfies the performance requirement. For example, the latency checking unit 153 determines that the performance requirement is satisfied if the calculated Index value is equal to or greater than the Index value specified as the performance requirement. If the latency inspection unit 153 satisfies the performance requirement, the processing proceeds to step S105. In addition, if the performance requirement is not satisfied, the latency checking unit 153 advances the processing to step S107.

  [Step S105] The behavior calculation unit 154 calculates the normal behavior of the container and the server, and stores them in the normal behavior storage unit 130. For example, the behavior calculation unit 154 acquires the metric values for the most recent predetermined period between the container and the server from the metric information storage unit 120, and calculates the percentile values for a plurality of percentile types. Then, the behavior calculation unit 154 stores the container behavior management table in which the percentile value of the container is set in the normal behavior storage unit 130 as information indicating the normal behavior of the container. Further, the behavior calculation unit 154 stores the server behavior management table in which the percentile value of the server is set in the normal behavior storage unit 130 as information indicating the normal behavior of the server.

  [Step S106] The performance adjustment engine 150 resets a variable R indicating the number of repetitions to “0”. Thereafter, the performance adjustment engine 150 proceeds with the process to step S102.

  [Step S107] The behavior calculation unit 154 calculates the behavior when the container and the server are abnormal. For example, the behavior calculation unit 154 acquires the metric values for the most recent predetermined period between the container and the server from the metric information storage unit 120, and calculates the percentile values for a plurality of percentile types. The percentile value calculated for each of a plurality of containers is information indicating the behavior of the corresponding container when it is abnormal. In addition, the percentile value calculated for each of the plurality of servers is information indicating the behavior when the corresponding server is abnormal.

  [Step S108] The abnormality factor estimation unit 155 calculates, for each metric type, a difference in behavior between the normal state and the abnormal state of the container that executes the component used to provide the performance adjustment target service. For example, the abnormality factor estimation unit 155 acquires a weighted percentile value from the normal behavior storage unit 130. Next, the abnormal factor estimation unit 155 compares the weighted percentile value indicating the behavior at normal time with the percentile value indicating the behavior at the time of abnormality calculated in step S107, and determines the positive factor degree and negative value for each metric type. Calculate the factor of

  [Step S109] The abnormality factor estimation unit 155 estimates a factor component based on the calculation result in step S108. For example, the abnormality factor estimation unit 155 extracts the factor value having the largest value from the positive factor value and the negative factor value for each metric type. Then, the abnormal factor estimation unit 155 estimates a component executed in the container from which the extracted factor degree is calculated as a factor component.

  [Step S110] The performance adjustment engine 150 determines whether or not the value of the variable R indicating the number of repetitions has reached a threshold value X (X is an integer of 1 or more). When the number of repetitions reaches the threshold value X, the performance adjustment engine 150 gives up performance adjustment and ends the processing. If the number of repetitions has not reached the threshold value X, the container arrangement control unit 156 advances the process to step S111.

  [Step S111] The container arrangement control unit 156 determines whether or not the factor degree code (container factor degree code) extracted in Step S109 is positive. If the container arrangement control unit 156 has a positive factor, the process proceeds to step S112. If the container placement control unit 156 has a negative factor, the process advances to step S113.

  [Step S112] The container placement control unit 156 scales out the factor component. That is, the container placement control unit 156 additionally places a container that executes the factor component on any server. For example, the container placement control unit 156 places a container on a server having the largest amount of free resources after placement among servers that can place the container. Thereafter, the container arrangement control unit 156 advances the processing to step S115.

  [Step S113] The container arrangement control unit 156 determines whether or not the server factor degree code is positive. If the server factor degree code is positive, the container arrangement control unit 156 advances the process to step S114. In addition, when the server factor degree code is negative, the container arrangement control unit 156 gives up performance adjustment and ends the processing.

  [Step S114] The container arrangement control unit 156 changes the arrangement of containers. That is, the container arrangement control unit 156 changes the arrangement destination of the container that is the source of calculation of the factor extracted in step S109 from the current server to another server.

[Step S115] The performance adjustment engine 150 increments the value of the variable R indicating the number of repetitions by 1, and advances the process to step S102.
In this way, it is possible to appropriately determine which component is a bottleneck in a service that does not satisfy the performance requirement, and perform performance adjustment so that the processing capability of the component is improved. Thereby, even if the performance requirement for each component is not defined, the component function is automatically expanded when the performance of the component is insufficient. As a result, for example, system operation management costs are reduced. Moreover, by automatically adjusting the performance of the component, it is not necessary to be aware of the performance of the component when developing the component, and the development cost is reduced.

  In the second embodiment, the component causing the latency deterioration is determined based on the difference in behavior between the normal state and the abnormal state of the container. This makes it possible to appropriately determine the component that causes the latency deterioration.

  Moreover, in the second embodiment, the percentile value is obtained from the metric frequency distribution, whereby the state indicated by the metric frequency distribution is replaced with an easily comparable numerical value. As a result, the difference in behavior between normal time and abnormal time can be quantified, and the container having the largest difference in behavior can be easily identified from among a plurality of containers.

  In the second embodiment, the weighted percentile value is used to strongly reflect the latest state with respect to the normal state. As a result, normal behavior can be calculated correctly. That is, in the cloud computing system, system configuration changes such as addition of servers and addition of software are frequently performed. For this reason, the normal behavior of containers and servers in the past in the past may be significantly different from the recent normal behavior. Further, if the behavior in the short period of time is regarded as the normal behavior, a special factor (for example, server failure) that occurs at a certain time is reflected in the behavior, and the accuracy as the normal behavior is lacking. Therefore, the performance adjustment engine 150 strongly reflects the recent normal behavior, and calculates the normal behavior based on the behavior over a relatively long period. As a result, the accuracy of normal behavior is improved.

  In the second embodiment, the performance adjustment engine 150 scales out the component corresponding to the container if the sign of the factor of the container (container factor sign) that is a factor of performance deterioration is positive. However, if the container factor degree sign is negative, the arrangement is changed. When the container factor degree sign is negative, the performance of the container that is the cause of performance degradation is not due to the problem of the container itself, but due to the problem of the server in which the container is installed (for example, overload due to execution of other software). There is a possibility. Therefore, the performance adjustment engine 150 moves the container from a server having some problem to another server by changing the arrangement of the container so that the container can exhibit its performance correctly. Thereby, excessive consumption of resources due to useless scale-out is suppressed.

[Third Embodiment]
Next, a third embodiment will be described. In the third embodiment, if scale-in is possible after scale-out, scale-in is performed.

  That is, the main purpose is to satisfy the performance requirements, but it is also important to achieve this with as few resources as possible. If you simply scale out, the amount of resource consumption increases, and resources that are originally unnecessary may be used. Therefore, in the third embodiment, in order to suppress an increase in unnecessary resource usage, the performance adjustment engine 150 performs scale-in after scale-out if possible.

  Specifically, if there are two servers with a smaller load than the server on which the factor component container is running, the performance adjustment engine 150 deletes the currently running container and Run the container on the server. If the total component load (total container load) after this scale-out (scale-up of 2 increase 1 decrease) is smaller than the normal total load, the performance adjustment engine 150 of the server on which the container is operating Select the server with the smallest load, and delete the container on the selected server. As a result, the performance is adjusted so as to satisfy the performance requirement without increasing the number of containers.

Hereinafter, the performance adjustment processing procedure according to the third embodiment will be described in detail with reference to FIGS. 22 to 24.
FIG. 22 is the first half of a flowchart showing an example of the procedure of the performance adjustment process in the third embodiment. Among the processes shown in FIG. 22, steps S201 to S204 and steps S206 to S210 are the same as the processes of steps S101 to S109 in the second embodiment shown in FIG. The process of different step S205 is as follows.

  [Step S205] If it is determined in step S204 that the performance requirement is satisfied, the container arrangement control unit 156 performs a scale-in process. When the scale-in process ends, the container arrangement control unit 156 advances the process to step S206.

FIG. 23 is a flowchart illustrating an example of the procedure of the scale-in process. In the following, the process illustrated in FIG. 23 will be described in order of step number.
[Step S221] The container arrangement control unit 156 determines whether or not the value of the flag “SCALE_FLAG” indicating that the scale-out of 2 increments and 1 decrements has been performed is “true”. The initial value of the flag “SCALE_FLAG” is “false”, and is updated to “true” after the scale-out of 2 increase 1 decrease is performed. If the value of the flag “SCALE_FLAG” is “true”, the container arrangement control unit 156 proceeds with the process to step S222. If the value of the flag “SCALE_FLAG” is not “true”, the container arrangement control unit 156 ends the scale-in process.

  [Step S222] The container placement control unit 156 determines whether or not the total load of the factor component when the scale-out of 2 and 1 is performed is less than or equal to the normal total load. The total load of the factor component is, for example, the sum of the latest metric values of the metric type having the largest factor degree at the time of scale-out of the container executing the component. The total load at the normal time is, for example, the total of the metric values in the past normal operation of the metric type having the largest factor degree at the scale-out of the container executing the factor component. If the total load of the factor component is equal to or less than the normal total load, the container arrangement control unit 156 advances the process to step S223. Further, if the total load of the factor component is larger than the normal total load, the container arrangement control unit 156 advances the processing to step S224.

  [Step S223] The container placement control unit 156 scales in the factor component. That is, the container arrangement control unit 156 deletes one of the containers that execute the factor component from the server. Thereafter, the scale-in process ends.

[Step S224] The container arrangement control unit 156 sets the flag “SCALE_FLAG” to “false”. Thereafter, the scale-in process ends.
In this way, the scale-in process is performed.

FIG. 24 is the latter half of the flowchart showing an example of the procedure of the performance adjustment process in the third embodiment. In the following, the process illustrated in FIG. 24 will be described in order of step number.
[Step S231] The container arrangement control unit 156 determines whether the value of the flag “SCALE_FLAG” is “true”. If the value of the flag “SCALE_FLAG” is “true”, the container arrangement control unit 156 proceeds with the process to step S232. If the value of the flag “SCALE_FLAG” is not “true”, the container arrangement control unit 156 advances the process to step S233.

  [Step S232] The container placement control unit 156 performs scale-out to increase the container for executing the factor component by one. Thereafter, the container arrangement control unit 156 advances the processing to step S241.

  [Step S233] The performance adjustment engine 150 determines whether or not the value of the variable R indicating the number of repetitions has reached the threshold value X. When the number of repetitions reaches the threshold value X, the performance adjustment engine 150 gives up performance adjustment and ends the processing. If the number of repetitions has not reached the threshold value X, the container arrangement control unit 156 advances the process to step S234.

  [Step S234] The container arrangement control unit 156 determines whether or not the factor degree code (container factor degree code) extracted in step S210 is positive. If the container arrangement control unit 156 has a positive factor, the process proceeds to step S237. If the container placement control unit 156 has a negative factor, the process advances to step S235.

  [Step S235] The container arrangement control unit 156 determines whether or not the server factor degree code is positive. If the server factor degree code is positive, the container arrangement control unit 156 proceeds with the process to step S236. In addition, when the server factor degree code is negative, the container arrangement control unit 156 gives up performance adjustment and ends the processing.

  [Step S236] The container arrangement control unit 156 changes the arrangement of containers. That is, the container arrangement control unit 156 changes the arrangement destination of the container that is the calculation source of the factor degree extracted in step S210 from the current server to another server.

  [Step S237] The container arrangement control unit 156 determines whether or not the server factor degree code is positive. If the server factor degree code is positive, the container arrangement control unit 156 proceeds with the process to step S238. If the server factor degree code is negative, the container arrangement control unit 156 advances the process to step S240.

[Step S238] The container arrangement control unit 156 performs a scale-out process of 2 increase 1 decrease.
[Step S239] The container arrangement control unit 156 sets the flag “SCALE_FLAG” to “true”. Thereafter, the container arrangement control unit 156 advances the processing to step S241.

[Step S240] The container arrangement control unit 156 performs a scale-out process by one increment.
[Step S241] The performance adjustment engine 150 increments the value of the variable R indicating the number of repetitions by 1, and advances the processing to Step S202 (see FIG. 22).

  In this way, when the scale-up is increased by 2 and 1 and the scale-in is possible, the scale-in can be performed. As a result, it is possible to effectively use resources without consuming resources unnecessarily.

[Other Embodiments]
In the second and third embodiments, the positive factor and the negative factor are calculated for each container. For example, the sum of the positive factor and the negative factor is calculated as the factor of the container. It may be a degree.

In the second and third embodiments, the percentile value is used as the representative value of the resource metric information, but other representative values such as an average value and a median value may be used.
As mentioned above, although embodiment was illustrated, the structure of each part shown by embodiment can be substituted by the other thing which has the same function. Moreover, other arbitrary structures and processes may be added. Further, any two or more configurations (features) of the above-described embodiments may be combined.

DESCRIPTION OF SYMBOLS 1 Service 2-4 Server 5 Terminal apparatus 10 Management apparatus 11 Memory | storage part 11a 2nd state information 12 Processing part

Claims (9)

On the computer,
Acquire performance information indicating the performance of services provided by linking multiple processes,
Determining whether the performance information satisfies a performance requirement indicating performance required for the service;
If the performance information does not satisfy the performance requirements, obtain first state information indicating the operation state of each of the plurality of processes in the latest predetermined period,
When the performance requirement is satisfied based on the second state information indicating the operation state of each of the plurality of processes when the performance of the service satisfies the performance requirement, and the first state information; Calculating the difference in operating state from when not satisfied for each of the plurality of processes;
Based on the difference between the operation states of each of the plurality of processes, the process that causes the performance deterioration of the service is determined.
Performance management program that executes processing. In addition to the computer,
When the performance information satisfies the performance requirement, third state information indicating an operation state of each of the plurality of processes in the latest predetermined period is acquired, and the second state information is obtained based on the third state information. Update,
The performance management program according to claim 1, wherein the process is executed. In the update of the second state information, based on the third state information of a plurality of periods, the operation state indicated by the third state information in a period closer to the present is more strongly reflected in the updated second state information. ,
The performance management program according to claim 2. The second state information is a predetermined representative value of second resource information indicating a time-series change in the operating status of resources used by each of the plurality of processes when the performance of the service satisfies the performance requirement. Is a second representative value,
In the acquisition of the first state information, a predetermined representative value of the first resource information indicating a time-series change in the operation status of the resource used by each of the plurality of processes in the most recent predetermined period is used as a first representative As a value,
In the calculation of the difference between the operating states, the difference between the first representative value and the second representative value is calculated for each of the plurality of processes.
The performance management program according to any one of claims 1 to 3. In addition to the computer,
Based on the difference in operating state of the factor processing determined as the performance deterioration factor, determine a method for dealing with the performance deterioration,
Implementing the countermeasure according to the determined countermeasure method,
The performance management program according to claim 1, wherein the process is executed. In the determination of the coping method, when the second state information of the factor processing represents an operating state having a load greater than the first state information of the factor processing, the factor processing is used as the coping method. If the first state information of the factor processing indicates an operating state having a larger load than the second state information of the factor processing, the factor processing is performed as the coping method. Decide to change the server running
The performance management program according to claim 5. In the determination of the coping method, execution of the factor processing at the first server that is currently executing the factor processing is stopped, and the factor processing is executed at each of a plurality of second servers different from the first server. To decide
In addition to the computer,
If the processing load for the plurality of second servers to execute the factor processing after performing the countermeasure according to the determined countermeasure method is less than or equal to a predetermined value, the factor processing in a part of the plurality of second servers Stop running,
The performance management program according to claim 5, wherein the process is executed. Computer
Acquire performance information indicating the performance of services provided by linking multiple processes,
Determining whether the performance information satisfies a performance requirement indicating performance required for the service;
If the performance information does not satisfy the performance requirements, obtain first state information indicating the operation state of each of the plurality of processes in the latest predetermined period,
When the performance requirement is satisfied based on the second state information indicating the operation state of each of the plurality of processes when the performance of the service satisfies the performance requirement, and the first state information; Calculating the difference in operating state from when not satisfied for each of the plurality of processes;
Based on the difference between the operation states of each of the plurality of processes, the process that causes the performance deterioration of the service is determined.
Performance management method. Stores second state information indicating the operating state of each of the plurality of processes when the performance of the service provided by linking the plurality of processes satisfies the performance requirement indicating the performance required for the service A storage unit;
Obtaining performance information indicating the performance of the service, determining whether the performance information satisfies the performance requirement, and when the performance information does not satisfy the performance requirement, First state information indicating the operation state of each process is acquired, and based on the first state information and the second state information, the operation state when the performance requirement is satisfied and when the performance requirement is not satisfied A processing unit that calculates the difference of each of the plurality of processes, and determines a process that is a factor that degrades the performance of the service, based on a difference in operation state of each of the plurality of processes.
A management device.

Images