JP2004302751A - Performance management method for computer system and computer system for managing performance of storage device - Google Patents

Abstract

In a computer system on which a DBMS operates, performance management for a storage device can be performed using a performance index in a user's business, thereby facilitating the performance management.
A management server of a computer system monitors an operation state of each component, a response time of a task, and the like, and executes a process using information related to a predetermined process such as a required performance and collected monitor information. When it is determined that tuning is necessary based on the monitor value, an instruction to change the allocation of the processing amount in the port, the allocation of the cache amount to the data, and the logical configuration of the disk device is issued.
For a process for performing a batch operation, a method for estimating the processing time is given to the management server, and a setting change instruction is issued based on the estimated execution time. In the process of performing an online job, when the response time or throughput of the process does not meet the required performance, an instruction to change the setting of the portion with a high load is issued.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a performance management method for a computer system and a computer system for managing the performance of a storage device. In particular, a large-scale database management system (DBMS, Database Management System) according to the required performance of business program processing. The present invention relates to a computer system performance management method suitable for adjusting the performance of a storage device in an operating computer system and exerting the required system performance, and a computer system for managing the performance of a storage device.
[0002]
[Prior art]
In recent years, a database management system (DBMS), which is software for performing a series of processes and management related to a database (DB, Database), has become extremely important. Even when examining the software configuration, those using a DB as a base have become dominant.
[0003]
When using an AP program, there is a required performance determined for each task, and it is strongly required to keep it, and its management is extremely important in the management task of the computer system. Generally, the performance of a DBMS is greatly affected by the performance of accessing data. Since the data access performance is strongly affected by the operation status of the hardware resources included in the data access path, it is important to manage the performance in consideration of the operation status of the storage device.
[0004]
In Patent Literature 1, in order to facilitate performance analysis of hardware resources included in a data access path, data mapping between storage devices is acquired from a DBMS, and an operation state (operation rate) in each layer, a certain table, A technology for presenting the operation status of a management structure of another layer corresponding to a data management structure such as a file is disclosed. When a specific part is in an overloaded state, it is highly likely that the specific part is a cause of performance degradation. Therefore, in Patent Literature 1, an optimization plan that solves the problem by changing the storage location of data belonging to a certain management structure is created.
[0005]
On the other hand, there is a storage device having a function of optimizing performance by eliminating an overload state of hardware resources. Patent Literature 2 discloses a storage device that causes a computer to recognize a logical storage device, maps the data to a plurality of physical disk devices existing in the storage device and stores the data, and dynamically maps the logical storage device to the physical disk device. (Hereinafter, used in the sense of "executed without stopping other processes") to improve the access performance of the storage device. In this patent document 2, a specific physical disk device is placed in an overloaded state by moving a part of data stored in a physical disk device with a high operating rate (overloaded state) to another physical disk device. And optimize performance.
[0006]
In addition, when connecting to a plurality of computers via the same port using a Fiber Channel or the like, there is a storage device having a function of giving priority to access from a specific computer (for example, non-patented devices). Priority Access function described in Document 1).
[0007]
Further, many of the storage devices have a cache memory. Therefore, the access performance can be improved by improving the probability that the data exists in the cache (hit) at the time of data reading. Patent Literature 3 discloses a technique for improving a cache hit rate by optimizing area allocation when a cache area is divided for each data type. In this patent document 3, in a system for performing cache data replacement control using an LRU (Least Recently Used) replacement algorithm, information on which position in an LRU management list exists at the time of a cache hit is obtained, and the area is determined using the information. The cache hit rate at the time of allocation change is estimated, and the area division is optimized.
[0008]
[Patent Document 1]
U.S. Patent No. 6,035,306
[Patent Document 2]
JP-A-9-274544
[Patent Document 3]
U.S. Pat. No. 5,434,992
[Non-patent document 1]
Hitachi Data Systems, Inc., "Hitachi Freedom Storage (TM) Lightning 9900 (TM) V Series", pp. 10-28. 8-9, DISK-432-00b, October 2002
[0009]
[Problems to be solved by the invention]
The performance index of a user's job is often a processing time in the case of a batch job in which a large amount of data processing is performed collectively, and a response time or a processing throughput in the case of an online job. However, the performance index used / provided by the conventional technology is directly linked to the physical operation such as the operation rate of the resource in the computer system and the cache hit rate, and is not directly linked to the performance index from the viewpoint of the user. In many cases, the performance required by the user cannot be achieved unless a plurality of performance indicators provided by the conventional technology are examined and a performance bottleneck is found and eliminated from the performance indicators. At present, these are compensated by an administrator, and this is not a simple task and requires special skills.
[0010]
As described above, there are many portions in which the manager is involved in the current performance management. However, it is preferable to automate the management as much as possible from the viewpoint of eliminating human errors and reducing other management costs.
[0011]
SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems of the related art, and an object of the present invention is to perform performance management for a storage device in a computer system in which a DBMS operates using a performance index in a user's business. And to facilitate performance management.
[0012]
A further object of the present invention is to automate performance tuning processing of a storage device in a computer system on which a DBSM operates, and to reduce performance management costs.
[0013]
[Means for Solving the Problems]
The following is performed to achieve the object of the present invention.
[0014]
First, in order to grasp the operation state of the components of the computer system, the operation status of each component, the response time of the business, and the like are monitored, and the results are collected in the management server. The management server issues a setting change instruction to the storage device when performing processing using the collected monitor information and the predetermined information such as the required performance, or when it is determined that tuning is necessary based on the monitor value. . The contents of the setting change include the allocation of the processing amount in the port of the storage device, the allocation of the cache amount to the data, and the configuration change of the storage area of the disk device for storing the data.
[0015]
For Job (a management unit of a program that realizes processing in the management server) that performs a batch job, a method for estimating the processing time is given to the management server, and the processing time is estimated using the method. Using the estimation, the setting of the storage device is changed so that the processing is completed within the required time. The processing time is estimated in consideration of the data processing amount and the I / O processing amount of the currently available storage device. The method of estimating the processing time can be determined by actually executing the processing, confirming the operation status of each component of the computer system, and associating it with the processing content.
[0016]
For a job that performs an online job, the response time and throughput of the processing are measured, and when the required performance is not satisfied, an instruction to change the setting of the storage device is issued. A part that is a performance bottleneck is found by referring to the operation rate of the resource of the storage device, and a setting change instruction is issued so as to eliminate the part. Also, if necessary, a cache allocation change instruction is issued to improve access performance.
[0017]
When a plurality of Jobs are executed in parallel in a computer system, resources in a storage device are shared between different Jobs, which may cause a performance bottleneck. Therefore, a processing priority is set for each Job, and when a problem occurs due to resource sharing, a part of the processing amount used by the low-priority Job is assigned to the high-priority Job. Is performed to maintain the required performance of Job, which is high.
[0018]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 28.
[0019]
[Embodiment 1]
Hereinafter, a first embodiment according to the present invention will be described with reference to FIGS.
(I) Configuration of computer system to which the present invention is applied
First, the configuration of a computer system that manages the performance of a storage device according to the first embodiment of the present invention will be described with reference to FIG.
FIG. 1 is a configuration diagram of a computer system that manages the performance of a storage device according to the first embodiment of the present invention.
[0020]
The computer system performs a storage device 40, a computer (hereinafter, “server”) 70 that uses the storage device 40, a computer (hereinafter, “management server”) 120 that manages system performance and the like, and a storage area virtualization process. It has a virtualization switch 60. Each device has a network I / F 22, and is connected to a network 24 via the network I / F 22, and can communicate with each other.
[0021]
The server 70, the virtualization switch 60, and the storage device 40 each have an I / O path I / F 32, and are connected by a communication line (hereinafter, “I / O path”) 34 via the I / O path I / F 32. I / O processing between the server 70 and the storage device 40 is performed using the I / O path 34. Note that the I / O path 34 may use a physical medium that differs between devices or a communication line that performs data transfer using a different protocol. Further, the network 24 and the I / O path 34 may be the same communication line.
[0022]
The storage device 40 has a CPU 12, a memory 14, a disk device (hereinafter, “HDD” (Hard Disk Drive)) 16, a network I / F 22, and an I / O path I / F 32, which are connected by an internal bus 18. You. The HDD 16 may be singular or plural. The memory 14 has a non-volatile area (ROM area) and a high-performance area (RAM area).
[0023]
In this embodiment, the storage device provides a logical disk device (hereinafter, “LU” (Logical Unit)) to an external device, and uses a block as a data access unit.
[0024]
A control program 44, which is a program for controlling the storage device 40, is stored in a nonvolatile area of the memory 14, and is executed by the CPU 12 after being moved to a high-performance area of the memory 14 at the time of startup. All the functions of the storage device 40 are realized and controlled by the control program 44.
[0025]
The memory 14 stores management information 46 used by the control program 44 to control and manage the storage device 40. Further, a part of the memory 14 is allocated to a data cache 42 which is an area for temporarily storing data requested to be accessed by an external device.
[0026]
The storage device 40 virtualizes the physical storage area of the HDD 16 and provides one or more LUs 208 to an external device. The LU 208 may correspond to the HDD 16 on a one-to-one basis, or may correspond to a storage area including a plurality of HDDs 16. Further, one HDD 16 may correspond to a plurality of LUs 208. The correspondence is held in the management information 46 in the form of the area mapping information 300. The storage device 40 has a function of dynamically creating / deleting the LU 208 and expanding / reducing the storage area of the LU 208 (see the definition in the section of the related art). Further, the storage device 40 has a function of dynamically changing the correspondence between the LU 208 and the storage area of the HDD 16 that stores the data by data movement.
[0027]
The storage device 40 has one or more ports 26 to the I / O path 34 of the I / O path I / F 32 that can access the LU 208. Further, it has a port to the network 24 in the network I / F 22.
[0028]
The correspondence between the port and the data to be accessed is held in the management information 46 in the form of higher-level port mapping information 322 (described later in FIG. 4). Further, the storage device 40 has a function of dynamically adding / deleting the port 26 to which the LU 208 can be accessed.
[0029]
In the storage device 40, storage areas are grouped in units of LUs 208, and an independent area in the data cache 42 is allocated to each group (hereinafter, this LU 208 group is referred to as a "cache group"). Creation / deletion of a cache group and addition / deletion of an LU 208 belonging to the cache group can be performed dynamically. Further, the storage device 40 has a function of dynamically changing the allocation amount of the data cache 42 to the cache group. The cache data replacement control in the cache group is performed independently using the LRU replacement algorithm. Also, regarding the storage area of the data that no longer exists in the data cache 42 due to the data replacement by the above algorithm, the information is stored in the management list for the LRU replacement algorithm as a virtual management area for a certain amount even after the replacement, and the operation is performed. Used to measure the situation.
[0030]
The storage device 40 has a function of executing an access request from a specific external device preferentially for each port 26 of the I / O path I / F 32 so that the processing amount per unit time satisfies the set value (hereinafter referred to as “priority”). Access control function "). The setting is held in the management information 46 in the form of port throughput setting information 440 (described later in FIG. 8). The storage device 40 has a function of creating, changing, and deleting the setting of the priority access control function in the port 26.
[0031]
The control program 44 measures the operating status of each existing part of the storage device 40 and stores it in the management information 46 in the form of storage monitor information 360. In particular, regarding the cache hit number statistics, the management list of the cache segment (area management unit of the data cache 42) for the LRU replacement algorithm is divided into a plurality of equal-sized areas, and the number of hits is measured for each divided area. Further, the virtual management area is also divided into areas of the same size as the real management area (the area where the cache segment is actually allocated to the data) and the number of hits is measured similarly to the real management area.
[0032]
The storage device 40 has a function of transmitting area mapping information 300, storage monitor information 360, port throughput setting information 440, and other configuration information of the storage device 40 to the outside via the network 24 in response to a request from the outside. . It should be noted that all of the data structures described here will be described later in detail.
[0033]
In addition, it has a function of executing the above-described various functions according to an instruction received from an external device via the network 24 from outside.
[0034]
The virtualization switch 60 has a CPU 12, a memory 14, a network I / F 22, and an I / O path I / F 32, which are connected by an internal bus 18. The memory 14 has a non-volatile area (ROM area) and a high-performance area (RAM area).
[0035]
A control program 64, which is a program for controlling the virtualization switch 60, is stored in a non-volatile area of the memory 14, and is executed by the CPU 12 after being moved to a high-performance area of the memory 14 at the time of startup. All functions provided by the virtualization switch 60 are controlled by the control program 64. The memory 14 also stores management information 66 used by the control program 64 to control and manage the virtualization switch 60.
[0036]
The virtualization switch 60 recognizes the LU 208 provided from the storage device 40 connected to the present device, virtualizes the storage area, and provides the virtual volume 206 to the external device. When the virtualization switch 60 is connected in multiple stages, the virtual volume 206 provided by the other virtual switch 60 is treated equivalently to the LU 208 provided from the storage device 40, and the storage area is virtualized to To an external device. The correspondence is held in the management information 66 in the form of area mapping information 300. The virtualization switch 60 has a function of dynamically creating / deleting the virtual volume 206 and expanding / reducing the storage area of the virtual volume 206. Further, it has a function of dynamically changing the correspondence between the LU 208 constituting the virtual volume 206 and the storage area of another virtual volume 206. Further, the virtualization switch 60 has a function of deleting an area corresponding to a free entry 314 described later in the area mapping information 300 held by the virtualization switch 60 from the area mapping information 300 (hereinafter, “release area”).
[0037]
The virtualization switch 60 has one or more ports 26 of the I / O path I / F 32 that can access the virtual volume 206. The correspondence is held in the management information 66 in the form of the upper port mapping information 322 (described later). Further, it has a function of dynamically adding / deleting a port 26 to which the virtual volume 206 can be accessed.
[0038]
When the virtual volume 206 provided by the virtualization switch 60 is accessed, the number of accesses performed, the type of the access, and the amount of transferred data are measured for each port 26 of the I / O path I / F 32, and the information is managed. It is stored in the form of port monitor information 400 (described later with reference to FIG. 9).
[0039]
The virtualization switch 60 has a function of transmitting the area mapping information 300 and the port monitor information 360 to the outside via the network 24 in response to a request from the outside. In addition, it has a function of executing the above-described various functions according to an instruction received from an external device via the network 24 from outside. These data structures will be described later in detail.
[0040]
The server 70 has a CPU 12, a memory 14, an HDD 16, a network I / F 22, and an I / O path I / F 32, which are connected by an internal bus 18.
[0041]
On the memory 14, an operating system (OS, Operating System) 72 and a management agent 144 are read from the HDD 16 and executed by the CPU 12.
[0042]
The OS 72 includes a device driver 76, a volume manager 78, and a file system 80. The OS 72 read into the memory 14 has OS management information 74 which is management information used by these and other programs constituting the OS 72. The OS management information 74 includes information on the hardware configuration of the server 70, such as the number of CPUs 12. In addition, the OS 72 has a software interface for an external program to read information stored in the OS management information 74. In the example shown in FIG. 1, the server 70 has only one file system 80, but may have a plurality of file systems 80. The OS 72 manages programs executed on the computer on which the OS 72 operates. At this time, how much the entire program being executed has used the CPU 12 is measured, and the operation rate of the CPU 12 is calculated based on the measurement, and the calculated operation rate is stored in the OS management information 74.
[0043]
The device driver 76 is a program that controls hardware such as the network I / F 22 and the I / O path I / F 32. When data transfer (access) is performed through the network I / F 22 or the I / O path I / F 32, the number of times of execution, the type thereof, and the amount of transferred data are determined by the network I / F 22 or the I / O path I / F 32. Is measured for each port 26 and stored in the OS management information 74 in the form of port monitor information 400.
[0044]
The volume manager 78 is a program that virtualizes the storage area of the LU 208 provided by the storage device 40 and the virtual volume 206 provided by the virtualization switch 60, and provides the logical volume 204 to the file system 80. The correspondence is held in the OS management information 74 in the form of area mapping information 300. The volume manager 78 has a function of dynamically changing the configuration (including creation and deletion) of the logical volume 204, and can give an execution instruction through a software interface of the volume manager 78. The volume manager 78 has a function of releasing an area corresponding to the free entry 314 based on the area mapping information 300 held by the volume manager 78. Further, the volume manager 78 may have a load distribution function of I / O processing using a plurality of I / O paths 34.
[0045]
The file system 80 virtualizes the storage areas of the LU 208 provided from the storage device 40, the virtual volume 206 provided from the virtualization switch 60, and the logical volume 204 provided from the volume manager 78, and converts the file 202 to another program. It is a program to provide. The correspondence is held in the OS management information 74 in the form of area mapping information 300. Note that the file system 80 also provides a raw device function that directly accesses the storage areas of the logical volume 204, the virtual volume 206, and the LU 208 with the same software interface as the file 202.
[0046]
The management agent 144 executes on the server 70 the processing request received via the network 24 from the system management program 140 executed on the management server 120, and outputs the result as needed through the system management program 140 via the network 24. This is a program that sends a reply. The processing executed by the management agent 144 includes 1) reading information stored in the OS management information 74, 2) execution / stop of the DBMS 90 and the AP program 100, and 3) information stored in the DBMS management information 92. 4) A SQL (Structured Query Language) statement is given to the DBMS 90 to obtain an SQL execution plan, 5) reading of information stored in the AP program management information 102, 6) a configuration change instruction to the volume manager 78, 7) Various setting instructions to the OS 72, and the like.
[0047]
The DBMS 90 is a program executed on the server 70 that executes a series of processes and management related to the DB. The DBMS 90 is started by the management agent 144 in accordance with an instruction from the system management program 140, read out from the HDD 16 or the storage device 40 in the server 70 to the memory 14, and executed by the CPU 12.
[0048]
The DBMS 90 uses a “data area” including a storage area provided by one or more files 202 as a management unit of the storage area. Tables, indexes, logs, and the like (hereinafter, collectively referred to as “data structures”) used and managed by the DBMS 90 each have a data area to be stored. Part of the data area may not be allocated to the data structure.
[0049]
The DBMS 90 read on the memory 14 has DBMS management information 92 which is management information of the DBMS 90 including the data storage area information 342. The DBMS 90 has a software interface for reading the DBMS management information 92 by an external program.
[0050]
Given an SQL statement, the DBMS 90 creates an execution plan for the statement (hereinafter, referred to as “SQL execution plan”), and then executes processing according to the plan. Basically, when the same SQL statement is given except for the search condition, the same SQL execution plan is created. The DBMS 90 has a software interface that outputs an SQL execution plan of a given SQL statement. Further, when outputting the SQL execution plan, the number of processes and the like estimated by the DBMS 90 may be output at the same time.
[0051]
The AP program 100 is a program executed on the server 70 for a task performed by a user, and issues a processing request to the DBMS 90. The AP program 100 is started by the management agent 144 in accordance with an instruction from the system management program 140, is read from the HDD 16 or the storage device 40 in the server 70 to the memory 14, and is executed by the CPU 12. The AP program 100 read into the memory 14 has AP program management information 102 as its management information. The AP program 100 may always issue a processing request to the DBMS 90 when handling data stored in the storage device 40. In this case, the server 70 on which the AP program 100 is executed / F32 may not be provided.
[0052]
The AP program 100 executed for online business in the present embodiment is realized as a set of one or more processes, and a process ID 432 as an identifier is assigned to each process. The user requests execution of any of the processes, and the AP program 100 performs the processes accordingly. The AP program 100 executes queuing control of the received processing request, and when the AP program 100 issues a processing request to the DBMS 90, the processing can be started immediately by the DBMS 90. The AP program 100 acquires the execution statistical information related to the processing, and holds it in the AP program management information 102 as online Job monitor information 430 (described later in FIG. 10). At this time, the AP program 100 has a software interface for an external program to read the AP program management information 102.
[0053]
A plurality of DBMSs 90 and AP programs 100 can be executed simultaneously on one server 70. Further, the DBMS 90 and the AP program 100 may be executed on different servers 70. In this case, the AP program 100 transmits a processing request to the DBMS 90 via a network.
[0054]
The OS 72, the DBMS 90, the AP program 100, and the management agent 144 are read from a CD-ROM (storage medium) storing them using the CD-ROM drive 20 of the management server 120, and stored in the server 70 via the network 24. Installed in the HDD 16 or the storage device 40.
[0055]
The management server 120 has a CPU 12, a memory 14, an HDD 16, a CD-ROM drive 20, and a network I / F 22, which are connected by an internal bus 18.
[0056]
The OS 72 and the system management program 142 are read from the HDD 16 onto the memory 14 and executed by the CPU 12. The CD-ROM drive 20 is used for installing various programs.
[0057]
Further, an input terminal 112 such as a keyboard and a mouse and a management terminal 110 having a display screen 114 are connected to the management server 120 via a network 24. This connection may use a communication line different from the network 24, and the management server 72 and the management terminal 110 may be configured integrally. The administrator inputs and outputs information through the management terminal 110 in principle, and also uses the CD-ROM drive 20 and the like as necessary.
[0058]
The system management program 140 is a program that implements performance and other system management functions of the management server 120. The data is read from the HDD 16 onto the memory 14 and executed by the CPU 12. The system management program 140 has system management information 142 which is management information necessary to realize the function. The CD-ROM storing the program is read out using the CD-ROM drive 20 of the management server 120 and installed on the HDD 16. Details of the operation of this program will be described later.
[0059]
In the example of the present embodiment, the system management program 140 is described as being executed on the management server 120. However, the system management program 140 may be executed on any server 70, the virtualization switch 60, and the storage device 40. At this time, in the server 70, the data is stored in the HDD 16 and read out to the memory 14, and then executed by the CPU 12. The virtualization switch 60 and the storage device 40 are stored in a non-volatile area of the memory 14 and executed by the CPU 12 after being moved to a high-performance area of the memory 14.
(II) Hierarchical structure of data mapping
Next, a hierarchical configuration of data mapping of data managed by the DBMS 90 of the present embodiment will be described with reference to FIG.
FIG. 2 is a hierarchical configuration diagram of data mapping of data managed by the DBMS 90 according to the first embodiment of the present invention.
[0060]
In the example of the present embodiment, a case where one virtualization switch 60 exists between the server 70 and the storage device 40 will be described. In the following, of the two hierarchies, the one closer to the DBMS 90 (ie, the application side) is referred to as “upper”, and the one closer to the HDD 16 (ie, the physical storage means) is referred to as “lower” hierarchy. The storage device 40, the virtualization switch 60, the volume manager 78, and the file system 80 are collectively referred to as a “virtualization mechanism”. The file 202, the logical volume 204, the virtual volume 206, and the LU 208 are collectively referred to as a "virtual structure", and the one obtained by adding the HDD 16 to the virtual structure is collectively referred to as a "management structure".
[0061]
In the example shown in FIG. 2, the DBMS 90 accesses the file 202 provided by the file system 80 storing the data structure 200 managed by the DBMS 90. The file system 80 converts access to the file 202 to access to a corresponding area of the logical volume 204. The volume manager 78 converts access to the logical volume 204 to access to a corresponding area of the virtual volume 206. The virtualization switch 60 converts access to the virtual volume 206 to access to the corresponding LU 208 area. The storage device 40 converts access to the LU 208 to access to the corresponding HDD 16. In this way, the virtualization mechanism maps the data of the virtual structure provided to the upper hierarchy to the storage area of one or more management structures existing in the lower hierarchy.
[0062]
Although not shown, the same part of data of a certain virtual structure may be mapped to a plurality of lower-level management structures. Further, there may be a plurality of paths on which data of a certain virtual structure is mapped to the HDD 16. In these cases, the virtualization mechanism holds such mapping in the area mapping information 300. Further, a management structure may have a mapping that is shared by multiple servers 70.
[0063]
In the present embodiment, the data correspondence between the management structures in the logical layer 212 only needs to be clarified, and the server 70 need not use the volume manager 78. Furthermore, the virtualization switch 60 may exist in a plurality of stages, or the server 70 and the storage device 40 may be directly connected by the I / O path 34 without the virtualization switch 60. The switch corresponding to the virtualization switch 60 may not have the storage area virtualization function. In this case, the virtualization switch 60 provides the management structure provided from the lower hierarchy as it is to the upper hierarchy as a virtual structure. Think there is.
[0064]
Note that, in the hierarchical configuration model of the present embodiment, if mapping is created so that only one data structure 208 is stored in one LU 208, an effect estimation when a configuration change of the storage device 40 described later is changed is calculated. There is a merit that it is useful for improving accuracy.
(III) Data structure
Next, a data structure used in the performance management method according to the present embodiment will be described with reference to FIGS.
[0065]
First, the area mapping information 300 will be described with reference to FIG.
FIG. 3 is a schematic diagram showing the data structure of the area mapping information 300.
[0066]
The area mapping information 300 is placed in the management information of each virtualization mechanism, and holds the correspondence between the virtual structure area provided by the virtualization mechanism and the management structure area used by the virtualization mechanism. , 304.
[0067]
In the description here, the upper virtual structure and the lower management structure are modeled as corresponding to each other, but actually, (file, logical volume), (logical Volume, virtual volume), (virtual volume, logical disk device), and (logical disk device, HDD).
[0068]
The entry 302 is information relating to the area of the virtual structure provided to the upper hierarchy by the virtualization mechanism. The entry 302 holds the virtual structure ID 306 which is the identifier of the virtual structure, the entry indicating the area in the structure, and the area is the lower hierarchy. When the HDD 16 is stored in a plurality of virtual structures or corresponds to the HDD 16 through different paths, the entry has a set of entries indicating a multiplexing method for identifying them. The entry 304 is information relating to the area of the lower-level management structure corresponding to the entry 302, and is an entry holding a port ID 326 that is an identifier of the port 26 of the I / O path I / F 32 used when accessing the management structure. , An entry holding a virtualization mechanism ID 308 which is an identifier of a virtualization mechanism that provides a management structure, an entry holding a management structure ID 310 which is an identifier of a management structure, and an entry indicating an area in the structure. When the file system 80 uses the logical volume 204 provided by the volume manager 78, it does not hold the entry holding the port ID 326. The storage device 40 does not hold an entry having the port ID 326 and the virtualization mechanism ID 308.
[0069]
A free entry 314 whose virtual structure ID 306 is “Free” indicates a storage area of a lower hierarchy that is available for use by the virtualization mechanism having the present area mapping information 300 but is not yet used as a virtual structure. I will show you. The virtualization mechanism having the function of dynamically changing the correspondence relationship by data movement uses a storage area of the management structure corresponding to the free entry 314 as a data movement destination.
[0070]
In the present embodiment, different virtual structures are allowed to use the storage area of the same management structure. Further, it is assumed that the virtualization mechanism ID 308, the virtual structure ID 306, the management structure ID 310, and the port ID 326 are identifiers uniquely determined in the system. Alternatively, even if it is not unique within the system, it is uniquely determined within the system by adding an identifier of the device.
[0071]
Next, the upper port mapping information 322 will be described with reference to FIG.
FIG. 4 is a schematic diagram showing the data structure of the upper port mapping information 322.
[0072]
The upper port mapping information 322 is placed in the management information 66 of the virtualization switch 60 and the management information 46 of the storage device 40, and is information for managing a virtual structure provided to an external device through a certain port. A pair of an entry to be held and an entry to hold the corresponding virtual structure ID 306 is held.
[0073]
Next, the data storage area information 342 will be described with reference to FIG.
FIG. 5 is a schematic diagram showing the data structure of the data storage area information 342.
[0074]
The data storage area information 342 is placed in the DBMS management information 92 of the server 70, is information used for storage area management of data managed by the DBMS 90, and an entry holding a data area name 344 which is the name of a data area. It consists of a set of entries holding a data structure name 346 which is the name of the data structure, and an entry holding a data storage location 348 which is information on where the corresponding data is stored in the file 202.
[0075]
Next, the cache group information 460 will be described with reference to FIG.
FIG. 6 is a schematic diagram showing the data structure of the cache group information 460.
[0076]
The cache group information 460 is placed in the management information 46 of the storage device 40, is information used by the storage device 40 for managing a cache group, and an entry holding a cache group ID 462, which is an identifier of the cache group, It has a set of entries holding the number of allocated segments 464, which is information on the number of cache segments allocated to the group, and a set of entries holding the LUID 364, which is the identifier of the LU 208 belonging to the cache group.
[0077]
Next, the storage monitor information 360 will be described with reference to FIG.
FIG. 7 is a schematic diagram showing the data structure of the storage monitor information 360.
[0078]
The storage monitor information 360 is placed in the management information 46 of the storage device 40 and holds data on the operation status of the storage device 40, and includes cache monitor information 362, storage port monitor information 382, and HDD monitor information 392. In.
[0079]
The cache monitor information 362 is operation statistical information regarding which cache hit has occurred when the LU 208 was accessed from an external device.
[0080]
The cache monitor information 362 includes a real area division number 502 indicating the number of divisions of the real management area, and a virtual management area number 504 indicating the holding amount of the virtual management area in units of the divided size of the real management area. Further, as a statistical value of the number of cache hits for each LU 208, an entry holding the LU ID 364, an entry holding the read / write I / O type 366 corresponding to the entry, and a cumulative execution count 368 of the I / O processing corresponding to the entry are stored. A set of entries to be held and an entry to hold a cache hit count total 370 indicating the number of hits at that time are held. The cumulative cache hit count 370 holds the total cumulative value of the number of hits in the real management area and the cumulative value of the number of hits in each area divided by both the real management area and the virtual management area. In the present embodiment, a serial number is added to an area obtained by integrating the real management area and the virtual management area, and the area where the most recently used data exists is the first area. It is assumed that this is an area having data.
[0081]
The storage port monitor information 382 is execution statistical information indicating how much data transfer processing has been performed using the port 26 of the I / O path I / F 32 for each LU 208, and is an entry holding a port ID 384 and an entry holding an LU ID 364. , And a pair of an entry having the cumulative execution number 368 of the I / O processing corresponding thereto and an entry holding the cumulative data transfer amount 386.
[0082]
The HDD monitor information 392 is execution statistical information indicating how much the HDD 16 is used for each LU 208. An entry holding the HDD ID 394, which is the HDD identifier, an entry holding the LU ID 364, and the actual processing corresponding thereto is performed. A set of entries holding an accumulated processing time 396 which is an accumulated value of time (not including the waiting time) is held.
[0083]
Next, the port throughput setting information 440 will be described with reference to FIG.
FIG. 8 is a schematic diagram showing the data structure of the port throughput setting information 440.
[0084]
The port throughput setting information 440 is set in the management information 46 of the storage device 40, and is setting information for a priority access control function for limiting an access processing amount to the specific port 26 from the specific port 26 of the external device. is there. The port throughput setting information 440 includes a set of an entry holding the port ID 384, an entry holding the access source port ID 442, an entry holding the maximum data transfer amount 444, and an entry 446 holding the maximum I / O processing amount 444. Have.
[0085]
The access source port ID 442 is the port ID of the port 26 of the external device that accesses the port 26 to be restricted. The maximum data transfer amount 444 is a maximum transfer amount set value per unit time corresponding to the port 26. The maximum I / O processing amount 444 is a set value of the maximum I / O processing amount per unit time.
[0086]
Next, the port monitor information 400 will be described with reference to FIG.
FIG. 9 is a schematic diagram showing the data structure of the port monitor information 400.
[0087]
The port monitor information 400 is placed in the management information 66 of the virtualization switch 60 and the OS management information 74 of the server 70, is execution statistical information on data transfer processing performed by the port 26, and is an entry holding the port ID 384. And an entry holding the corresponding I / O type 366, an entry having the cumulative execution number 368 of the I / O processing, and a response time which is a time (including a waiting time) from issuance / reception of the processing request to completion of the processing. It has a set of entries holding an accumulated response time 402 which is an accumulated value and an entry holding an accumulated data transfer amount 386.
[0088]
Next, the online Job monitor information 430 will be described with reference to FIG.
FIG. 10 is a schematic diagram showing the data structure of the online Job monitor information 430.
[0089]
The online Job monitor information 430 is placed in the AP management information 102 of the server 70 and is execution statistical information for the online Job acquired by the AP program 100. The entry holding the process ID 432 and the cumulative execution of the corresponding process It has a set of entries holding the number of times 368, entries holding the cumulative response time 402 of the process, and entries holding the cumulative processing time 396 when the process is issued to the DBMS 90.
[0090]
Next, the storage performance information 610 will be described with reference to FIG.
FIG. 11 is a schematic diagram showing the data structure of the storage performance information 610.
[0091]
The storage performance information 610 is information held in the system management information 142 of the management device 120, and has HDD performance information 612 and port performance information 616. This information is created by acquiring the model of the storage device 40 and its corresponding part, and combining it with the performance information for each model given in advance.
[0092]
The HDD performance information 612 is information relating to the access performance of the HDD 16, an entry holding the device ID 572 which is an identifier of the storage device 40, an entry holding the HDD ID 394, and the access performance information corresponding to the access performance of the HDD 16. 612, including a set of entries.
[0093]
The access performance information 612 holds the average response times when a cache hit / no (miss) occurs during a read at random access and when a cache hit / does not occur during a write.
[0094]
The port performance information 616 is information on the maximum performance of the port 26 of the I / O path I / F 32 of the storage device 40, and includes an entry holding the device ID 572, an entry holding the port ID 384, and the corresponding port performance information 618. It has a set of entries. The port performance information 612 holds a set of a maximum I / O processing amount (IOPS) and a maximum data transfer amount (MB / s) as information of the maximum performance of the port.
[0095]
Next, the monitor history information 510 will be described with reference to FIG.
FIG. 12 is a schematic diagram showing the data structure of the monitor history information 510.
[0096]
The monitor history information 510 is information held in the system management information 142 of the management device 120, is monitor information collected by each device or program, and includes the cache monitor information 370, storage port monitor information 382, The HDD monitor information 392, the port monitor information 400, the online job monitor information 430, and the operating rate of the CPU 12 calculated by the OS 72 (hereinafter collectively referred to as "monitor information") are obtained and edited into an appropriate form. The stored information is stored as history information.
[0097]
A monitor part ID 514 uniquely determined in the system is added to a part whose operation status is monitored (the cache monitor information 370, the storage port monitor information 382, and the HDD monitor information 392 also include the access destination LU 208).
[0098]
The monitor history information 510 has a set of an entry holding the monitor part ID 514, an entry holding information indicating the monitor content 516, an entry holding the corresponding previous acquisition information 518, and an entry holding the history information 520. . The specific contents of the monitor content 516 are as described when describing the data structure of each monitor information.
[0099]
The previously obtained information 518 has an entry of the date and time when the value of the monitor information was previously obtained and an entry of the obtained value.
[0100]
The history information 520 is composed of an entry holding a history content 522 indicating the content of a value that is totaled and stored as a history, and a plurality of entries holding a sample 524 that is a monitor value for a certain period. The history contents 522 in the present embodiment include the average execution count and the average data transfer amount for each LU 208 in the port 26 of the I / O path I / F 32 of the storage device 40 from the storage port monitor information 382, and the cache monitor information 370 to the LU 208 The average number of executions and the average actual hit rate for each read / write, the average hit rate for each read area (the number of read hits in the area in the period / the number of read executions to the LU 208 in the period), the HDD monitor information and the HDD 16 From the average operation rate for each LU 208 and the port monitor information 400, the average for each read / write in the port 26 of the I / O path I / F 32 of the server 70 and the virtualization switch 60 or the port 26 of the network I / F of the server 70 Number of executions, average data transfer amount, average response time, online The average number of executions of each processing each as the ob monitor information 430 at the AP program 100, the average response time, average processing time, there is a CPU utilization rate in the server 70 and from the CPU12 uptime OS72 is calculated.
[0101]
The sample 524 includes an entry indicating when the data stored therein is a monitor value, and an entry holding an average value and a maximum value, which are total values in the period.
[0102]
Next, the batch job management information 700 will be described with reference to FIG.
FIG. 13 is a schematic diagram showing the data structure of the batch job management information 700.
[0103]
The batch Job management information 700 is management information for the batch Job placed in the system management information 142 of the management server 120, an entry holding the Job ID 622 that is the identifier of the Job, and a Job priority indicating the execution priority of the Job. An entry holding the degree 704, an entry holding the Job type 706, an entry holding an execution condition 708 indicating under what start condition this Job is executed, and a maximum execution which is a setting value of the maximum execution time of the Job. A set of an entry holding a time 710, an entry holding an execution server ID 712 which is an identifier of the server 70 executing the job, and an entry holding an execution command 714 which is a command executed by the server 70, Job-dependent input information 720, Job-dependent output data information 730, CPU processing time estimation Act 716, Job is information about the data used including I / O processing content information 740a. A plurality of pairs of the execution server ID 712 and the execution command 714 may exist.
[0104]
In the case of the batch job management information 700, the job type 706 holds information indicating that the job is for a batch job. The contents of the execution condition 708 include, in the batch job management information 700, an execution start time including a condition of a date to be executed and a day of the week, or execution when one or more different jobs are completed.
[0105]
As an execution image of the Job of the present embodiment, it is assumed that the batch Job is executed continuously and the output data of the preceding job is taken over as the input data of the next job.
[0106]
The job-dependent input information 720 is information relating to data used when executing the job, and includes an entry holding the preceding job ID 722 which is the job ID 622 of the job outputting the data to be used, and data information which is information of the input data. It is composed of a pair of an entry holding 724 and an entry holding a data amount unit 726 which is information of a data unit of a data amount such as the number of lines of data and a file size.
[0107]
The information stored in the data information 724 is basically an identifier of data stored in the storage device 40, such as an identifier of the file 202 or a set of an identifier of the DBMS 90 and the data structure 200 managed by the DBMS 90. It may be an identifier for delivery. Note that when the input-dependent process is not performed, the job-dependent input information 720 does not exist.
[0108]
The job-dependent output data information 730 is information relating to job output data used for executing another job. An entry holding the next-stage JobID 732 which is the JobID 622 of the job using the output data, and data information of the output data 724 and a set of entries holding the data amount unit 726. The output data amount returned as the processing completion information at the end of the job is measured using the data amount unit 726 in the job-dependent output data information 730. If there is no Job using the output data, the job-dependent output data information 730 does not exist.
[0109]
The CPU processing time estimating method 716 is information on a method of estimating a time during which the CPU 12 of the server 70 is a bottleneck in the processing execution time.
[0110]
The I / O processing content information 740a is information relating to data used by the Job. An entry holding the data information 724 of the data used by the Job and an entry holding the access order 742 of the corresponding data, I / O An entry holding the O-neck 744, an entry holding the I / O type 746, an entry holding the I / O base performance 748, and an entry holding a processing amount estimation 752 which is information on a method of estimating the processing amount from the input data amount. , A set of entries holding storage hints 754 which are contents to be instructed to the storage device 40 when executing Job. Although the storage hint 754 may have various contents regarding the storage device 40, in the present embodiment, the cache amount for data is explicitly set.
[0111]
The access order 742 is information on the access order, and means that the smaller the number is, the earlier the access is made, and that the same value is accessed simultaneously. The I / O neck 744 is information indicating whether the I / O processing becomes a performance bottleneck or not, and takes values of “Y” and “N”. The I / O type 746 is information on the I / O processing type, and takes a value of “sequential” or “random”. The I / O base performance 748 is information on I / O performance used for estimating execution time and the like, and is used when accessing the data in order to cope with distributed arrangement of the data in a plurality of storage devices 40 and the like. A set of a port ID 384 and a performance value is held for each port 26 of each storage device 40 to be executed. The performance value is sequential performance (data transfer speed) when the I / O type 746 is “sequential”, and random performance (IOPS) when it is “random”. The processing amount estimation 752 holds the information for each port ID 384 held in the I / O base performance 748.
[0112]
Next, the batch SQL design information 850 will be described with reference to FIG.
FIG. 14 is a schematic diagram showing the data structure of the batch SQL design information 850.
[0113]
The batch SQL design information 850 is input to the management server 120 in order to set the batch job management information 700, and is information relating to a process requested by the AP program 100 to the DBMS 90. The batch SQL design information 850 corresponds to an entry holding the Job ID 622 of the Job performing the setting, an entry holding the execution DBMS ID 852 which is an identifier of the DBMS 90 performing the processing, an entry holding the main input data 854, and an entry holding the SQLID 858a. It has a set of entries holding the execution SQL statement 860 to be executed, a set of entries holding the data information 724 and the corresponding entry holding the SQLID 858b, and the repetition execution SQL information 862.
[0114]
The main input data 854 is data whose amount has the greatest effect on the CPU processing time of the AP program 100.
[0115]
The SQLID 858a is assigned a serial number in the order of execution by the AP program 100, and as the information of the SQL statement issued by the AP program 100 to the DBMS 90, the SQLID 858 and the execution SQL statement 860 which is the SQL statement to be executed are held correspondingly.
[0116]
The SQLID 858b is the data information 724 when the file 202 is used as the input / output data, and the ID of the SQL statement that uses the data information and saves the result.
[0117]
The repetition execution SQL information 862 is information on an SQL statement to be repeatedly executed, and has a set of an entry holding an SQLID set 864 and an entry holding data information 724 of input data at the time of the repetition. The SQLID set 864 is a set of SQLIDs 858 of one or a plurality of SQL statements that are repeatedly executed.
Next, the online job management information 770 will be described with reference to FIG.
FIG. 15 is a schematic diagram showing the data structure of the online Job management information 770.
[0118]
The online Job management information 770 is placed in the system management information 142 of the management server 120 and is information for managing the online Job. An entry holding the Job ID 622, an entry holding the Job priority 704, and a Job type 706 are provided. , An entry holding the execution condition 708, an entry holding the required performance 774, a set of an entry holding the execution server ID 712 and an entry holding the execution command 714, the I / O processing content information 740c, and the number of processes. Includes an entry that holds the processing multiplicity 782 indicating the severity.
[0119]
In the present embodiment, the batch Job management information 700 and the online Job management information 770 shown in FIG. 13 are separated in consideration of the fact that the required performance and the information to be managed are different depending on the nature of the Job. Things.
[0120]
In the case of the online job management information 770, information indicating that the job type 706 is for online business is held. A plurality of pairs of the execution server ID 712 and the execution command 714 may exist. As the contents of the execution condition 708, the online Job management information 770 gives an execution start time and an execution end time (including an execution at all times) including a condition of a date to be executed and a day of the week. The required performance 774 is setting information of the required performance of the Job, and holds the average response time of the entire processing and the processing throughput (average number of executions of the processing) in a high load state.
[0121]
The I / O processing content information 740c is obtained by extracting a part of the I / O processing content information 740a from the batch job management information 700 shown in FIG. , And a pair of an entry holding the corresponding I / O base performance 748 and an entry holding the storage hint 754. Data accessed by the process is grasped from the job design information, data information 724 is set, and the port ID 386 of the I / O base performance 748 is obtained by referring to the mapping aggregation information. As for the storage hint 754, instruction information for the storage device is set. In the present embodiment, an instruction to calculate a data cache amount desired to be used for each data and set the value is set as needed.
[0122]
The performance value of the I / O base performance 748 is always random performance (IOPS) in the I / O processing content information 740c. This value is determined, for example, by actually executing the processing, and grasping the processing throughput and the I / O performance of the port 26 of the I / O path I / F 32 of the storage device 40 from the monitor information of the storage device 40 at that time. Then, the I / O performance required to achieve the throughput given to the required performance 774 is estimated as the throughput is proportional to the I / O performance, and the value is set. If a plurality of data are stored in the same LU 208 via the same port 26, the necessary performance is appropriately distributed among them.
[0123]
Next, the job resource management information 620 will be described with reference to FIG.
FIG. 16 is a schematic diagram illustrating the data structure of the Job resource management information 620.
[0124]
The job resource management information 620 is information used by the system management program 140 of the management server 120 as work information, and includes job port setting information 630 and job cache amount information 650.
[0125]
The job port setting information 630 is information on the setting of the Job using the port 26 and the priority access control function assigned to the Job for each port 26 of the I / O path I / F 32 of the storage device 40, and includes the device ID 572. , An entry holding a port ID 384, an entry holding a Job ID 622 of a Job accessing data using the corresponding port 26, and an entry holding a performance ratio 632 assigned to the port 26 of the Job. Have a pair. The performance ratio 632 is a performance index value calculated as a ratio from the maximum performance of the corresponding port held in the port performance information 616, and the conversion into the I / O processing amount or the data transfer amount is performed on the port 26. The processing is performed by referring to the corresponding port performance information 618 to obtain the maximum I / O processing amount and the maximum data transfer amount of the port 26 and multiplying the obtained data by the performance ratio 632.
[0126]
The job cache amount information 650 is, for each cache group of the storage device 40, information on the Job using the LU 208 belonging to the cache group and the amount allocated to the Job, and holds the entry holding the device ID 572 and the cache group ID 462. It is composed of a set of an entry, an entry holding the Job ID 622 of a Job using data stored in the LU 208 belonging to the corresponding cache group, and an entry holding allocation information 652, which is information on a cache allocation amount to the Job.
[0127]
As for the quota, it is possible to have information of a plurality of generations managed in the form of a stack. The reason why the quota is set in the quota information 652 is also held in combination with the quota, and when the cause is solved. Should be able to return to the old settings. When the cache allocation amount of one or more cache groups used by Job is changed, the generation is updated in synchronization with the other cache groups used by Job whose cache amount is not changed.
[0128]
Next, the priority control information 880 will be described with reference to FIG.
FIG. 17 is a schematic diagram showing the data structure of the priority control information 880.
[0129]
The priority control information 880 is setting information used when the system management program 140 in the management server 120 performs control in consideration of the job priority, and corresponds to an entry holding the job priority 704. A set of entries holding the margin coefficient 882 is included.
[0130]
In the present embodiment, the job priority 704 is divided into P levels, and takes 1, 2,... The margin coefficient 882 is a coefficient for estimating a margin for the estimated data when determining a method of changing the setting of the storage device 40. The higher the priority of Job, the larger the margin. In the embodiment, one or more values are taken.
(IV) Performance management method processing
Next, the processing of the performance management method according to the present embodiment will be described with reference to FIGS.
[0131]
(IV-1) Outline of processing of system management program
First, the processing outline of the system management program 140 will be described.
[0132]
The system management program 140 grasps the configuration related to the system. An identifier is added to each of the storage device 40, the virtualization switch 60, the server 70, the volume manager 78, the file system 80, the DBMS 90, and the AP program 100, which are constituent elements, to manage them. Further, the connection relationship between the network 24 and the I / O path 34 is held in the system management information 142.
[0133]
The system management program 140 acquires various information from other devices. At this time, with respect to the information held by the storage device 40 and the virtualization switch 60, a request for transmitting information is issued directly via the network 24, and the information is acquired. As for the information held by the program executed on the server 70, an information read request is issued to the management agent 144 via the network 24, and the necessary information is read to collect the information. .
[0134]
The system management program 140 acquires the area mapping information 300 from each virtualization mechanism, the data storage area information 342 from the DBMS 90, the upper port mapping information 322 from the storage device 40 and the virtualization switch 60, and the cache group information 460 from the storage device 40. These are stored in the system management information 142 as mapping aggregation information together with the identifier of the acquisition source.
[0135]
Here, in the management of the cache group of the storage device 40, the system management program 140 treats a specific cache group as a “free cache group” to be preferentially reduced by adjusting the cache allocation amount. The LU 208 having no cache amount allocation request is controlled as belonging to the free cache group.
[0136]
In addition, the real area division number 502 and the virtual management area number 504 included in the port throughput setting information 440 and the cache monitor information 362 are also acquired from each storage device 40 and stored in the system management information 142 together with the identifiers of the acquisition sources. I do. For these pieces of information, after the setting is changed or after a certain period of time, the information is reacquired and the consistency of the held contents is confirmed.
[0137]
In more detail, as the mapping aggregation information of the system management information 142 described above, the system management program 140 holds information on mapping related to the hardware resources of the running Job in the system management information 142. For the server 70, the identifier of the server 70 on which the Job is implemented and which data is accessed via which port 26; for the virtualization switch 60, the data is passed through which virtualization switch 60; Regarding which storage device 40 is accessed via which port 26 of which I / O path I / F 32 of the virtualization switch 60, which data uses which LU 208 of which storage device 40, and the LU 208 stores the data. Information regarding which port 26 of the I / O path I / F 32 of the device 40 is accessed, which HDD 16 stores the data in the LU 208, and which cache group the LU 208 belongs to. Hold. Hereinafter, this information is referred to as “Job resource mapping information”.
[0138]
(IV-2) Job Management Method of System Management Program and Setting of Values in Job Management Information
Hereinafter, a job management method of the system management program and setting of a value in the job management information will be described.
[0139]
The system management program 140 stores batch job management information 700 as job management information for performing batch operations, online job management information 770 as job management information for performing online operations in the system management information 142, Hold each. These pieces of information are stored in the HDD 16 and read out into the system management information 142 on the memory 14 when the system management program 140 is started.
[0140]
As described above, the reason why the job management information 700 and the online job management information 770 are separated is that the required performance and the information to be managed are different depending on the nature of the job.
[0141]
Hereinafter, when indicating either the batch Job management information 700 or the online Job management information 770, or both, it is referred to as “Job management information”. Job management information is basically given by an administrator, but some information is set by a method described later.
[0142]
Also, the I / O processing content information 740a in the batch Job management information 700 and the I / O processing content information 740c in the online Job management information 770 are collectively referred to as "I / O processing content information 740".
[0143]
As described above, in the execution of the job according to the present embodiment, the batch job is executed continuously, and the output data of the preceding job is taken over as the input data of the next job. In the Job managed by the batch Job management information 700, the output data amount of the data set in the Job dependent output data information 730 is returned as the processing completion information at the end of the Job. Then, based on this information and the information in the batch Job management information 700, the execution time of the continuously executed Job is estimated.
[0144]
Further, in the present embodiment, the output data amount uses what is output as the processing completion information by the AP program 100 executed by the Job for batch work.
[0145]
Hereinafter, the setting processing of the value of the batch Job management information 700 will be described.
[0146]
The batch job management information 700 is information given for each job of a batch job.
[0147]
In this entry, Job ID 622, Job priority 704, Job type 706, execution condition 708, maximum execution time 710, execution server ID 712, execution command 714, Job dependent input information 720, Job dependent output data information 730, the administrator This is information given before executing Job.
[0148]
A method for estimating the maximum execution time of Job will be described later in detail.
[0149]
The CPU processing time estimation method 716 and the I / O processing content information 740a are set by actually monitoring monitor information.
[0150]
Hereinafter, the processing for setting the CPU processing time estimation method 716 and the I / O processing content information 740a will be described with reference to FIG.
FIG. 18 is a flowchart of a process for setting the CPU processing time estimation method 716 and the I / O processing content information 740a in the job management information 700 by actually measuring the monitor information.
[0151]
At the start of the procedure, other information in the batch Job management information 700 and the batch SQL design information 850 shown in FIG. 14 are given by the administrator. At the start of this procedure, various data need to exist in the system as in the actual operation (step 1101).
[0152]
First, the DBMS 90 to be processed is grasped from the execution DBMS ID 852 in the batch SQL design information 850, and all the SQL execution plans of the SQL statements in the execution SQL statement 862 are acquired therefrom (step 1102).
[0153]
The SQL execution plan is information for executing an SQL statement, and includes processing steps for executing an SQL sentence, details of the processing, usage data and detailed information of an access method, and the like.
[0154]
In some cases, a table to be used temporarily temporarily is explicitly created during the process. To acquire an SQL execution plan of an SQL statement using the table, an SQL execution plan is acquired after the table is created. If possible, the process is executed step by step for each SQL statement, and an SQL execution plan is obtained immediately before the execution of the process. After acquiring all the SQL execution plans, delete the tables to be used temporarily.
[0155]
From the acquired SQL execution plan and the information in the batch SQL design information 850, the data accessed by each SQL statement, the access method, and the access order are grasped (step 1103).
[0156]
Since the SQL execution plan includes the processing steps for executing the SQL statement, the processing content thereof, the usage data and the detailed information of the access method, the access data, the access method, and the access order in one SQL statement are based on the SQL execution plan. We can understand from plan.
[0157]
As for the temporary table created implicitly by the DBMS 90 when performing the processing, the data area name 344 of the data area in which the table is created is grasped. From the SQLID 858 and the repetition execution SQL information 862 in the batch SQL design information 850, the execution order including repetition can be known. At this time, when one or more SQL statements are repeatedly executed, it is assumed that all data accessed by the statements are simultaneously accessed by the repetition processing.
[0158]
Then, the obtained access destination data and access order are set to the data information 724 in the I / O processing content information 740a, the access order 742, and the port ID 384 of the I / O base performance 748. If the same data is accessed in a plurality of different processing steps, they are independent. The port ID 384 can be obtained together with the storage device 40 in which it exists by referring to the mapping aggregation information in the system management information 140, and a plurality of port IDs 384 may be obtained.
[0159]
Here, when the grasped process is “full scan of the table”, the I / O type 746 is “sequential” (“Seq” in FIG. 13), and the I / O type 746 is “selection process using B-Tree index”. Is set to "random"("Rnd" in FIG. 13) for both the index and the table. All inputs and outputs to and from the file 202 are “sequential”.
[0160]
From now on, the amount of data input from the outside when performing the processing is grasped (step 1104).
[0161]
Regarding the data set in the job-dependent input information 720, the data amount is grasped in accordance with the data amount unit 726. In the present embodiment, in principle, a plurality of Jobs form a series of processes. Therefore, the jobs are sequentially executed and the amount of output data output in the system management information 142 as a completion report is grasped.
[0162]
Subsequently, the processing is actually executed, the execution time is measured, the CPU operation rate in the server 70 on which the DBMS 90 operates, and the I / O processing execution status (I / O processing execution count, data transfer amount) in the storage device 40. Is grasped (step 1105).
[0163]
The operating statuses of these are grasped by the above-mentioned system management program 140 collecting them. At the time of data acquisition, the data acquisition interval is set to be sufficiently smaller than the processing execution time. In order to reduce the system load, if possible, detailed monitor information is acquired only for the portion storing the CPU operation rate of the server 70 related to the processing execution and the data grasped in step 1103.
[0164]
From the obtained monitor information, the CPU utilization rate in the server 70 on which the DBMS 90 operates is grasped for a time that is equal to or greater than a certain threshold (for example, 80%), and the time is indicated by the main input data 854 in the batch SQL design information 850. The CPU processing time estimation method 716 is determined as being proportional to the data amount of data (step 1106).
[0165]
The input data amount used in step 1104 is used.
[0166]
Next, the I / O neck, I / O base performance, and I / O processing amount per input data amount of each access data are grasped from the monitor information, and the value of the I / O processing content information is set (step 1107). .
[0167]
That is, based on the access data grasped in step 1103, the access order, and the access destination port 26 of the storage device 40, the storage information is associated with the monitor information of the port 26 in the storage port monitor information 382 of the storage device 40.
[0168]
Thereafter, a monitor value of the CPU operation rate of the server 70 on which the DBMS 90 operates in a period corresponding to certain access data is checked. If the monitor value exceeds the threshold value in step 1106, it is determined that the I / O bottleneck is not present. The value of the I / O neck 744 is set to “Y”; otherwise, the value is set to “N”.
[0169]
Subsequently, the amount of I / O processing at the port 26 and the time during which the I / O processing was performed are ascertained for each LU 208 from the monitor information, and the I / O base performance 748 corresponding to the data is taken as the average performance for each port 26. Set to. At this time, when the value of the corresponding I / O type 744 is “sequential”, the data transfer amount is used as the processing amount, and when the value is “random”, the I / O processing execution count is used. Further, a corresponding processing amount estimate 752 is set from the input data amount and the I / O processing amount for each port 26 grasped in step 1104. When a plurality of different data are stored in the same LU 208 and they are accessed via the same port 26, the I / O processing amount is appropriately divided among them.
[0170]
Then, the procedure ends (step 1108).
[0171]
In this procedure, the storage hint 754 in the I / O processing content information 740a is not set. This is set by the administrator after the end of this procedure. When the storage hint 754 is set, since the I / O processing performance changes, steps 1105 to 1107 are executed again, and the I / O neck 744 and the I / O base performance 748 are reset.
[0172]
Next, a method of estimating the processing time of the maximum execution time 710 of a job managed by the batch job management information 700 will be described.
[0173]
In the present embodiment, the processing time of Job managed by the batch Job management information 700 includes the CPU processing time when the CPU is a bottleneck and the CPU processing time when the I / O is a bottleneck. It is estimated as the sum of the I / O processing time and the time affected by the I / O delay when the CPU is the bottleneck. The amount of data input to each Job is stored in the system management information 142.
[0174]
The CPU processing time when the CPU is in the bottleneck state is calculated from the CPU processing time estimation method 716 and the input data amount given at the start of the processing.
[0175]
The I / O processing time when the I / O is in the bottleneck state is the time when the processing is started for the data identified by the data information 724 in which the value of the I / O neck 444 in the I / O processing content information 740a is “Y”. It is calculated from the input data amount given at the time, the processing amount obtained by the processing amount estimation 752, and the I / O processing performance available for data access, and is obtained as the total sum.
[0176]
The processing performance available for accessing each data is obtained as follows. First, for each data accessed by Job, the port 26 of the I / O path I / F 32 of the storage device 40 used when accessing the data is obtained from the I / O base performance 748. Subsequently, the data is grouped according to the port 26 obtained for each data and the value of the access order 742 corresponding to the data, and a data set for simultaneously accessing the port 26 is obtained. In each group, the performance ratio 632 in which the Job can be used on the port 26 is stored in the Job port setting information 630 or specified as a separate condition. Then, the performance ratio 632 is distributed in proportion to the performance ratio 632 calculated from the performance value of the corresponding port 26 of the I / O base performance 748 corresponding to each data. In this way, the performance ratio 632 in which certain data can be used is obtained, and the I / O processing performance in the port 26 which can be used for accessing data is calculated.
[0177]
The time affected by the I / O delay when the CPU is the bottleneck is obtained as follows. Regarding the data identified by the data information 724 in which the value of the I / O neck 444 in the I / O processing content information 740a is “N”, first, the I / O is the same as the I / O processing time in the bottleneck state. First, the processing time at the current I / O processing amount setting is obtained. Subsequently, assuming that the I / O processing performance is the performance given to the I / O base performance 748, the I / O processing time is similarly obtained. The value obtained by subtracting the latter from the former for each port 26 of each data is the required influence time. When a negative value is obtained by calculation, the influence time is assumed to be zero.
[0178]
In the present embodiment, the input data amount is grasped from the job processing completion report of the preceding stage in principle. If there is no corresponding front-stage Job such as a Job that extracts data processed within a certain period of time during the execution of the AP program 100 that is constantly being executed, a job for acquiring the amount of input data to the Job is created and advanced. By executing the process and outputting the target amount of input data as a completion report, the amount of input data necessary for estimating the processing time is stored in the system management information 142. In the above-described example, the number of executions of the target process of the AP program 100 within the target period is acquired from the monitor history information 510 as the input data amount, and is used as the number of records. Instead of creating the data amount estimation Job, a method for acquiring the data input amount may be stored in the job-dependent input information 720, and the data amount may be grasped using the method.
[0179]
(IV-3) Collection of monitor information of the system management program and Job tuning processing based on the monitor information
Next, the collection of the monitor information of the system management program and the Job tuning process based on the monitor information will be described with reference to FIGS.
[0180]
The collection of the monitor information of the system management program and the Job tuning processing based on the monitor information are always executed in an infinite loop while the management server 120 is executing and the system management program monitors the system.
FIG. 19 is a general chart of collection of monitor information of the system management program and Job tuning processing based on the monitor information.
FIG. 20 is a flowchart of a Job tuning process based on monitor information (part 1).
FIG. 21 is a flowchart of a Job tuning process based on monitor information (part 2).
[0181]
The setting of the value in the monitor history information 510 is performed as follows.
[0182]
This process is started when the management server 120 is started up, the system management program 140 is started, and its initialization is completed (step 1401).
[0183]
First, as the initial monitor value, the value of the current monitor information is obtained from all devices or programs that have obtained the monitor information (step 1402), and the corresponding acquisition date and time and the entry of the acquired value in the previous acquisition information 518 are entered. Update.
[0184]
Subsequently, the process is put to sleep until the next data acquisition timing (step 1403). The acquisition of the data is performed at regular intervals after the previous monitor value acquisition or at a specific time determined in advance. At this time, the data acquisition interval may be individually managed according to the content to be monitored, and may be different for each content to be monitored.
[0185]
When the data acquisition timing comes, the current value of the monitor information and the current date and time are acquired. After the acquisition of the information, the previous acquisition date and time and the acquired value stored in the corresponding previous acquisition information 518 are grasped, and a method of calculating the value of the history content 522 is selected based on the monitor content 516. , And its value is calculated (step 1404).
[0186]
The period is grasped from the current date and time and the previous acquisition date and time, and the calculated value is stored in the entry holding the sample 524 as the average and maximum of the period. After that, the current date and time and the value of the monitor information acquired this time are stored in the previously acquired information 518 as the previously acquired date and time and the acquired value.
[0187]
Subsequently, a tuning process is performed based on the collected monitor information starting from step 1501 (step 1405, the processes in FIGS. 20 and 21).
[0188]
Then, the sample 524 holding the data before a certain time is arranged to reduce the data amount of the monitor history information 510 (step 1406).
[0189]
The method performs 1) combining a plurality of samples 524 within a certain period into one sample 524 according to the age of the data, and 2) deleting data of particularly old samples 524. After the processing is completed, the process returns to step 1403, and the processing is repeated thereafter.
[0190]
As the configuration of each device is changed, the number of parts to be monitored dynamically increases and decreases. If the number of monitored parts increases, an entry is added to the monitor history information 510 in step 1404, and the part is also monitored. If the monitor part is deleted, the corresponding data is not subjected to the processing of step 1404, and if all the samples 524 of the monitor part deleted in step 1406 are deleted, the entry in the monitor history information 510 is deleted. Is deleted.
[0191]
Next, a job tuning process based on monitor information will be described with reference to FIGS.
[0192]
The job tuning processing based on this monitor information is the processing shown in step 1405 of FIG.
[0193]
In this processing, the processing is performed for all the running Jobs at the same time, and the processing is performed in order from the job with the highest Job priority 704 in each step. In Steps 1515 and 1517, when a plurality of Jobs are required to move data or increase / decrease the I / O processing amount for the same part, only the Job having the highest Job priority 704 is processed. carry out. If it is determined that a plurality of problems exist in one Job, the process branches to a plurality of independent paths.
[0194]
The Job tuning process is called as a subroutine in Step 1405 of FIG. 19 (Step 1501).
[0195]
First, it is confirmed whether or not the change of the HDD overload state can be executed by the “data transfer request” stored in the system management 142 (step 1502). The change of the HDD overload state is to relocate the LU 208 to an appropriate HDD 16 by data movement. Note that the data movement request handled here has been executed in step 1508 (described later) of the previously executed Job tuning process.
[0196]
Here, the maximum values of the operating rates of the source HDD 16 and the destination HDD 16 in the last and a certain period (for example, 30 minutes) from the current time within a certain period (for example, one week) are stored in the monitor history information 510. It is determined that the data movement can be executed when the value is lower than a predetermined threshold value (for example, 40%) on both sides. Similarly, when there is a request for data movement in another virtualization mechanism, the operation rate of the related part is similarly examined, and when the value of all parts is less than a predetermined threshold, the data movement and the Change the settings accordingly. With respect to the data movement request determined to be executable, a processing request for realizing the data movement is issued to the related virtualization mechanism.
[0197]
Next, if the job currently uses a larger cache amount than the initial setting with respect to the data cache 42 of the storage device 40, it is checked whether or not it can be released (step 1503).
[0198]
Here, referring to the allocation information 652 in the Job cache amount information 650 shown in FIG. 16, an entry whose set amount is larger than the initial setting (data of the oldest generation) is searched from the newer generation, and the cause thereof is determined. Understand from the setting reason. Then, the latest data of the monitor history information 510 corresponding to the cause is grasped, and it is confirmed whether the load is less than a certain ratio (for example, 75%) of the load stored during the setting reason, and the load is reduced. If so, it is determined to return to the setting one generation before. If it is determined to return to the previous setting, the data of the older generation of the allocation information 652 is repeatedly confirmed.
[0199]
By this confirmation processing, the generation of the allocation information 652 to be returned is grasped, and if such information exists, the new generation data after the setting of the allocation information 652 in the job cache amount information 650 to be returned is discarded. Then, in accordance with the data of the returned generation, a change instruction of the cache allocation amount is issued to the storage device 40.
[0200]
Next, referring to the latest data of the monitor history information 510, it is confirmed whether or not there is a problem (step 1504).
[0201]
Here, when the job is managed by the online job management information 770, that is, when the job is an online job, each processing in the latest AP program 100 stored in the monitor history information 510 shown in FIG. The average response time, the average processing time, and the average number of executions are grasped, and then the average response time, the average operation rate, and the average number of executions of the entire AP program 100 are calculated. Then, referring to the corresponding online Job management information 770, the obtained average response time satisfies the average response time set in the required performance 774, and the average operation rate is (processing multiplicity 782) × (predetermined. If the average execution count satisfies the processing throughput set in the required performance 774, it is determined that the load is high, and there is a condition not satisfied. If there is a problem.
[0202]
When a job is managed by the batch job management information 700, that is, when the job is a batch job, the HDD 16 in which data used by the job is stored is determined by referring to the job resource mapping information, and the HDD 16 When the overall operation rate exceeds a predetermined threshold value (for example, 70%), it is determined that an overload state is present, and it is determined that there is a problem.
[0203]
If there is a problem, the process proceeds to step 1505. If not, the process proceeds to step 1530 and ends.
[0204]
Next, by referring to the latest data of the monitor history information 510 shown in FIG. 12, it is confirmed whether there is an overloaded part other than the storage device 40 for the hardware related to Job (step 1505).
[0205]
The hardware related to Job includes the CPU operation rate in the server 70, the port 26 of the I / O path I / F 32 in the virtualization switch 60 and the server 70, and the like, which are obtained by referring to the Job resource mapping information. be able to. The overload state is recognized as a state in which each operation rate exceeds a value predetermined for each hardware. If such a part exists, the process proceeds to step 1525; otherwise, the process proceeds to step 1506.
[0206]
If it is possible to reduce the load on the part determined to be in the overloaded state in step 1505, an instruction is issued to the OS 72 and the virtualization switch 60 in the server 70 to realize this ( Step 1525). Since the present invention is focused on adjusting the performance of the storage device, the specific method of step 1525 will not be described in detail.
[0207]
If it is determined in step 1505 that there is no overloaded part other than the storage device 40, it is determined whether there is an overloaded part related to the storage device 40 that stores data used by Job. Is confirmed (step 1506).
[0208]
The confirmation part includes the port 26 of the I / O path I / F 32 and the HDD 16. The HDD 16 is as described in step 1504.
[0209]
The port 26 is checked only when the Job is managed by the online Job management information 770. First, referring to the job resource mapping information held in the system management information 142, the port 26 used by the job and the LU 208 accessed through the port 26 are grasped. Using the latest data of the monitor history information 510 shown in FIG. 12, the calculated performance ratio 632 of the LU 208 at the port 26 is calculated, and the total sum is calculated for each port 26. If the calculated value exceeds a certain ratio (for example, 90%) of the performance ratio 632 of the port 26 allocated to the Job stored in the Job port setting information 630, the port 26 transmits the job to the Job. It is determined that there is an overload condition.
[0210]
If it is determined in step 1506 that the associated port 26 is overloaded with respect to Job, the process proceeds to step 1511 (FIG. 21A). If the associated HDD 16 is in an overloaded state, the process proceeds to step 1507. If all the related ports 26 and HDD 16 are determined to be normal values, the process proceeds to step 1519. After this step, if there are a plurality of parts in the overloaded state, the path is divided for each part, and the process proceeds for each part.
[0211]
If it is determined in step 1506 that all the related ports 26 and the HDD 16 are normal values, a method of changing the cache amount allocation of the cache group used by the job having the performance problem is determined (step 1519). ).
[0212]
In this process, for the Job that has come to this step, a process of tuning the cache amount for the online Job is performed in the descending order of the Job priority 704. The process of tuning the cache amount for online Job will be described later with reference to (IV-5) in FIG.
[0213]
At this time, when the cache group used by the previously tuned Job is grasped from the Job resource mapping information in the system management information 142, and the process of tuning the cache amount for the online Job for the subsequent Job is performed. Then, an instruction is issued so as not to reduce the cache quota of those cache groups. Note that, after allocating a fixed ratio (for example, 5%) of the current cache allocation amount to Job from the free cache group, a process of tuning the cache amount for online Job may be performed.
[0214]
Then, a change instruction of the cache allocation amount obtained in step 1519 is issued to the corresponding storage device 40, and the value of the job cache amount information 650 in the job resource management information 620 of FIG. 16 is updated according to the setting. (Step 1521).
[0215]
Then, the process ends (step 1530).
[0216]
If it is determined in step 1507 that the related HDD 16 is overloaded, Job checks whether it is possible to perform tuning for adding a cache to a cache group belonging to the LU 208 that stores data in the overloaded HDD 16. (Step 1507).
[0219]
For such a cache group, if the amount of cache of the job is to be increased by a certain percentage (for example, 5%) from the current value, refer to the job cache amount information 650 to see if the amount can be secured from the free cache group. Then, if the allocation is possible, the cache allocation change processing is performed. If not possible, use all possible amounts for this tuning. At this time, when different data used by Job are stored in the HDD 16 in question by different LUs 208, the average of the change in LU processing time used is considered, and the sum in the related LU 208 is maximized. assign. The process of creating a change plan for changing the cache allocation amount in consideration of the average of the LU processing time change will be described later with reference to FIG. 25 of (IV-5). When updating the allocation information 652 of the Job cache amount information 650 to the value set in this step, the allocation information 652 is stored as new generation data, and the HDDID 394 of the overloaded HDD 16 and the current operation rate are stored as the setting reason.
[0218]
Next, a data movement request is created to move the data of the LU 208 used by Job stored in the overloaded HDD 16 to the normally loaded HDD 16 (step 1508).
[0219]
The data movement request is created as follows. First, the overloaded HDD 16 is assumed to be the source HDD 16. By referring to the Job resource mapping information, the LU 208 used by the highest priority Job using the data among the LUs 208 stored in the source HDD 16 is grasped as the LU 208 to move the data. Then, the storage device 40 having the source HDD 16 and the LU 208 to which the source HDD 16 data is to be moved are specified, a process for determining a method of resolving the HDD underload condition is performed, a data transfer request is created, and the system management is performed. It is stored in the information 142. A method of resolving the HDD underload condition will be described later with reference to (IV-5) in FIG.
[0220]
After the data transfer request is created, the operation rate after execution of the data transfer request is estimated assuming that the operation rate of the entire HDD 16 decreases by the operation rate of the moved LU 208, and the value does not fall below a predetermined threshold value (for example, 60%). In this case, the LU 208 used by the job having the next highest priority is set as the LU 208 for moving data, and processing for determining a method for resolving the HDD underdeployment state is performed. Or until the confirmation is completed for all Jobs.
[0221]
If the virtualization switch 60 and the volume manager 78 can dynamically change the correspondence between the virtual structure and the management structure by data movement, if the creation of a data movement request in the storage device 40 fails, The data migration request in the virtualization switch 60 or the volume manager 78 may be created by applying the processing from the processing for determining the method of resolving the HDD underload condition mutatis mutandis. At this time, the following points are also considered. 1) Create a new virtual structure from the free area and move the data there. 2) When determining the destination, not only the operation rate of the HDD 16 but also the operating status of the port 26 in the access path (the route of the I / O path 34 from the server 70 to the storage device 40) for the data destination is checked, and The operation rate estimated as the value of is not to exceed 100%. This estimation is performed in the same manner as the estimation of the operation rate in the HDD 16. 3) In the storage device 40 as the transfer destination, the same cache amount as the current condition is allocated to the cache group to which the LU 208 holding the moved data belongs.
[0222]
Next, it is determined whether the cache amount allocated in step 1507 is sufficient (step 1509).
[0223]
Considering the average of the LU processing time change determined by the method for resolving the HDD overload condition, it is sufficient if the sum of the values in the related LU 208 is larger than a predetermined threshold value (for example, 5% of the unit time). It proceeds to step 1521. If not, the process proceeds to step 1517 as insufficient.
[0224]
If it is determined in step 1507 that the allocated cache amount is sufficient, the cache controller issues the cache allocation amount change instruction and the I / O performance setting change obtained so far to the corresponding storage device 40, and in accordance with the setting, The values of the job port setting information 630 and the job cache amount information 650 in the job resource management information 620 of FIG. 16 are updated (step 1521).
[0225]
If it is determined in step 1507 that the allocated cache amount is insufficient, it is determined here whether or not tuning is possible to reduce the I / O processing amount to the LU 208 in which Job stores data in the overloaded HDD 16. If possible, the setting of the I / O processing is changed (step 1517). This step needs to be performed after step 1515, which is performed asynchronously with this processing in another path when there is a problem with the port.
[0226]
In this process, first, the job priority 704 is obtained from the job management information of the job currently being tuned. The job using the LU 208 that stores data in the HDD 16 in the overloaded state is grasped by referring to the job resource mapping information in the system management information 124, and the job priority 704 of the job is obtained in the same manner as described above. The job whose priority is lower than the job priority 704 of the currently tuned job is obtained. For those Jobs, the port of the I / O path I / F 32 of the storage device 40 used when accessing the LU 208 that stores data in the overloaded HDD 16 is obtained from the Job resource mapping information, and the Job port setting information is obtained. The value of the performance ratio 632 of the Job stored in the port 26 at the port 26 is reduced by a predetermined ratio (for example, the current value of 5%). When the amount of access to the HDD 16 can be limited directly, the setting may be performed.
[0227]
Then, step 1521 is performed, and the process ends (step 1530).
[0228]
If it is determined in step 1506 that the associated port 26 is overloaded with Job, a method for resolving the case where the port 26 is overloaded with Job is determined (step 1511). First, at the port 26 which is overloaded with respect to Job, referring to the Job port setting information 630 in the Job resource management information 620 shown in FIG. The ratio 632 is obtained, and an I / O performance setting change method for increasing the value by a fixed ratio (for example, 5%) is created.
[0229]
Next, it is checked whether the I / O processing amount increase plan created in step 1511 can be realized (step 1512).
[0230]
When the I / O performance setting change method obtained in step 1511 is adopted, the performance ratio 632 assigned to each Job of the port 26 that is overloaded with Job is referred to the Job port setting information 630. Then, the sum is calculated, and if the value does not exceed 100%, the process proceeds to step 1521 as feasible. If it exceeds 100%, it is determined that it cannot be realized, and the process proceeds to step 1513.
[0231]
If it is determined in step 1512 that the I / O processing amount increase plan cannot be realized, the server 70 avoids the LU 208 accessed via the problem port 26 that is overloaded for Job. It is checked whether an overload state can be avoided by adding or changing an access path to be accessed (step 1513).
[0232]
This is realized by dynamically setting / changing a plurality of access paths by the volume manager 78, the virtualization switch 60, and the storage device 40, and is held in Job resource mapping information and other system management information 142. Confirm with information. If it is possible, the process proceeds to step 1523; otherwise, the process proceeds to step 1514.
[0233]
If it is determined in step 1513 that the access path can be changed or added, the server 70 sends the LU 208 accessed via the port 26 in question, which is determined to be created or changeable in step 1513, to the LU 208. An instruction is issued to the volume manager 78, the virtualization switch 60, and the storage device 40 so as to realize an access path for avoiding the access (step 1523), and the process is terminated (FIG. 20C). Step 1530).
[0234]
If it is determined in step 1513 that the access path cannot be changed or added, it is next checked by referring to the Job resource mapping information whether the port 26 that is overloaded with the Job is shared by a plurality of Jobs ( Step 1514).
[0235]
If it is shared, the process proceeds to step 1515. If it is not shared, it is determined that tuning is not possible, the process proceeds to step 1530, and the process ends.
[0236]
If it is determined in step 1514 that the port 26 in the overloaded state is shared by a plurality of Jobs, the port 26 having the highest Job priority 704 among the Jobs in question is An I / O processing performance setting change method for narrowing down the I / O processing amount of another Job sharing the port 26 is determined (step 1515).
[0237]
That is, it is assumed that the value of the performance ratio 632 currently assigned to Job at the problem port 26 is increased by a fixed ratio (for example, 5%), and the Job ID 622 of Job, the port ID 384 of the problem port 26, and the newly allocated performance ratio 632 are specified. Then, the process starting from step 1601 is performed, and the allocation of new I / O processing performance at the port 26 is determined.
[0238]
Then, the process proceeds to step 1521 (FIG. 20B).
[0239]
(IV-4) Job execution processing by the system management program
Next, Job execution processing by the system management program will be described with reference to FIG.
FIG. 22 is a flowchart of Job execution processing by the system management program.
[0240]
The system management program 140 refers to the execution condition 708 in the given Job management information and executes the Job when the given condition is satisfied (step 1301).
[0241]
At that time, the time at the start of processing is stored in the system management information 142 together with the Job ID 622. When jobs for a plurality of batch jobs are executed consecutively, the value of the maximum execution time 710 in the batch job management information 700 may be changed according to the processing time of the preceding stage. The value of is stored separately.
[0242]
When the job is started, first, a hardware resource used by the job is grasped and set in the job resource mapping information (step 1302).
[0243]
This is because the execution server ID 712 in the job management information identifies the server 70 on which the processing is performed, and the data to be used is grasped from the data information 724 in the I / O processing content information 740. Refer to the information.
[0244]
Next, a method of changing the I / O processing performance at the port 26 of the I / O path I / F 32 of the storage device 40 using the priority access control function is determined, and the setting instruction is given to the storage device 40 based on the determined method. Is issued (step 1303). The process of determining the method of changing the I / O processing performance will be described later in (IV-4).
[0245]
Here, under the condition that the allocation is added to the job to be started, the process of determining the method of changing the I / O processing performance is performed, and the obtained value is reflected in the job port setting information 630. The change plan obtained here is an assignment for each Job, and the I / O of the server 70 or the virtualization switch 60 that is an access source to the storage device 40 is utilized by using the Job resource mapping information in the system management information 142. The port 26 of the path I / F 32 is obtained, the values from the same port 26 are collected, the setting value for the storage device 40 is obtained, and the setting instruction is issued to the storage device 40.
[0246]
Next, the setting of the cache group and the method of setting and changing the cache amount are determined, and the setting instruction is issued to the storage device 40 based on the determination (step 1304). The process of determining the allocation of the cache amount when starting Job will be described later with reference to (IV-5) in FIG.
[0247]
Here, first, it is checked whether or not there is a cache amount allocation instruction from the storage hint 754 in the I / O processing content information 740 of the Job management information. Subsequently, with respect to the data for which there is an assignment instruction, the job resource mapping information is referred to, the LU 208 in which it is stored is checked, and it is checked whether it belongs to the free cache group. If there is one, the same data is stored, and the LU 208 provided by the same storage device 40 creates a configuration plan that forms the same cache group, and creates a new cache group if necessary. Then, the information is added to the Job resource mapping information. At this time, LUs 208 holding different data as much as possible belong to different cache groups.
[0248]
Subsequently, when the job management information and the cache group setting of the job to be started are changed, the information is given, the process starting from step 2301 is performed, and a value is set in the job cache amount information 650. The value obtained is the cache allocation amount for each Job, and the cache capacity used by a certain cache group is the sum of the allocation amounts given to Jobs using that cache group. An instruction is issued to the storage device 40 in accordance with those changes.
[0249]
After these processes, the server 70 identified by the execution server ID 712 in the Job management information is instructed to execute the corresponding execution command 714, and the process is executed (step 1305).
[0250]
After the processing is completed, in the case of a Job managed by the batch Job management information 700, the output data amount of the data set in the Job-dependent output data information 730 is acquired as the processing completion information, and the value is obtained as the Job ID 622 or its output. The data is stored in the system management information 142 together with the data information 724 for identifying the data.
[0251]
When the Job ends, the allocation of the I / O processing performance at the port 26 set in Step 1303 is released to another Job (Step 1306).
[0252]
Here, the same processing as in step 1303 is performed under the condition of deleting the execution-completed assignment to Job.
[0253]
Next, the settings of the cache group and the cache allocation set in step 1304 are released (step 1307).
[0254]
Here, the job ID 622 of the completed Job is given, and the process starting from step 2501 is performed to determine a cache group configuration change method. The contents are reflected in the job cache amount information 650 and the job resource mapping information, and a change instruction is issued to the storage device 40 according to the obtained method.
[0255]
Subsequently, execution information such as the job execution time (start / end time) and the number of processes is output as a job execution history (step 1308).
[0256]
Thereafter, the process ends (1309).
[0257]
When the maximum execution time 710 in the batch job management information 700 is changed at the start of the process, the original value stored separately is returned. When the processing of the job managed by the batch job management information 700 is completed, the execution condition 708 in the other batch job management information 700 is referenced to confirm whether there is a job to be executed following the completed job, This procedure is executed again to execute the job.
[0258]
(IV-5) Processing for Changing Settings of Specifications of Storage Device
Next, a process of changing and setting the specifications of the storage device 40 will be described with reference to FIGS.
[0259]
First, a process of determining the allocation of the cache amount when the system management program 140 starts a job will be described with reference to FIG.
FIG. 23 is a flowchart of a process in which the system management program 140 determines allocation of a cache amount when starting Job.
[0260]
The process for determining the allocation of the cache amount is called from step 1304 in FIG. 22 when the system management program 140 starts Job.
[0261]
At the start of the process, the job management information of the job to be executed and the cache group configuration plan set at the start of the execution of the job are given from the upper routine (step 2301).
[0262]
First, data used by the job and the storage device 40 in which the data is stored are obtained from the job resource mapping information (step 2302).
[0263]
Then, each cache request amount is grasped from the storage hint 754 in the I / O processing content information 740 in the job management information, and the cache request amount for each LU 208 is referred by referring to the job resource mapping information in the system management information 142. Find the quantity. When one data is stored in a plurality of LUs 208, it is assumed that the amount of data storage area in each LU 208 is confirmed from the mapping aggregation information, and the cache amount is divided among the LUs 208 in proportion thereto.
[0264]
When a plurality of data are stored in one LU 208, the sum of the data is assumed to be the cache request amount of the LU 208. The values are totaled for each cache group according to the configuration given at the start of the processing, and are stored in the Job cache amount information 650 as Job initial settings. Further, the values are totaled for each storage device 40 and set as a cache request amount.
[0265]
Subsequently, it is confirmed whether or not the cache request amount obtained in step 2302 can be allocated from the free cache group for each storage device 40. If the amount obtained by subtracting the cache request amount from the cache amount of the free cache group does not fall below a predetermined threshold, it can be allocated. If it is determined that allocation is impossible, the following processing is performed.
[0266]
First, the job priority 704 given to the job to which the cache is assigned is grasped from the job management information. Then, for the cache group of the storage device 40, the job using the job is grasped from the job resource mapping information, the job priority 704 of the job is obtained, and a job whose value is lower than the job assigned to the cache is obtained. . If there is no such Job, there is no change from the current assigned amount. If there is, a method of reallocating a part of those cache capacities to the Job requesting the cache allocation is determined (step 2303).
[0267]
For example, the cache allocated to a low-priority Job is reallocated to a job assigned to the cache in an amount proportional to the amount. At this time, for the Job whose cache amount has been reduced, a new generation of data is set in the allocation information 652 in the Job cache amount information 650, with the value after the set amount reduced and the set amount as “cache shortage”.
[0268]
Subsequently, an allocation method to the cache group is determined according to the allocation amount obtained in step 2303 (step 2304).
[0269]
When the requested amount can be obtained from the free cash group, the value becomes the allocated amount as it is. If it cannot be acquired, an amount proportional to the requested amount is assigned to each cache group so that the total sum becomes the assigned amount. At this time, in the allocation information 652 in the Job cache amount information 650, the data of the new generation, which is the amount actually allocated as the set amount and whose setting reason is “insufficient cache”, is set.
[0270]
Then, the process ends (step 2305). Note that a cache tuning process described later with reference to FIG. 25 may be performed on Job whose cache amount has been reduced.
[0271]
Next, with reference to FIG. 24, a description will be given of a cache reallocation process executed by the system management program 140 as a job termination process.
FIG. 24 is a flowchart of a cache reallocation process performed by the system management program 140 as a job termination process.
[0272]
This cache amount reassignment process is called from step 1307 in FIG. 22 when the system management program 140 ends Job.
[0273]
In this process, at the start of the process, the Job ID 622 of the Job completed from the upper routine is given (step 2501).
[0274]
First, referring to the job cache amount information 650, the cache amount used by the ending Job is grasped as an open cache amount, and the amount is totaled for each storage device 40 (step 2502).
[0275]
Then, the corresponding entry is deleted from the job cache amount information 650. Furthermore, referring to the Job resource mapping information, it is assumed that the cache group to which only the LU 208 in which only the data of the ending Job is stored belongs and the cache group is deleted.
[0276]
Subsequently, referring to the job cache amount information 650, the storage device 40 storing the data of the job to be ended recognizes the job to which the cache amount corresponding to the requested value is not allocated. The entry for which the setting reason of the corresponding allocation information 652 is “insufficient cache” corresponds to this. Then, the job priority 704 of those Jobs is grasped from the job management information, and the open cache is re-allocated in order from the job priority 704 having the highest job priority 704 so as to satisfy the shortage from the required value of the cache amount (step). 2503).
[0277]
In step 2503, if there are remaining caches in the open cache even after the caches have been reallocated to all Jobs, they are allocated to the free cache group (step 2504).
[0278]
Then, the process ends (step 2505). For the job cache group in which the shortage has been satisfied, the corresponding entry of the job cache amount information 650 is returned to the generation before the shortage.
[0279]
Next, a process of tuning the cache amount for online Job based on monitor information will be described with reference to FIG.
FIG. 25 is a flowchart of a process of tuning the cache amount for online Job based on monitor information.
[0280]
This process is a process called from steps 1507 and 1519 of the job tuning process in FIGS. 20 and 21.
[0281]
In this process, at the start of the process, a Job ID 622 of a Job to be tuned from a higher-level routine and a cache group whose cache allocation must not be reduced as necessary are given (Step 2701).
[0282]
First, the storage device 40, the LU 208, and the cache group that hold the data used by the designated Job are obtained from the Job resource mapping information (step 2702).
[0283]
Next, the amount of change in the cache hit rate when the cache amount is changed is calculated (step 2703).
[0284]
In this embodiment, when the values of the real area division number 502 and the virtual management area number 504 in the storage device 40 are R and V, respectively, and the current cache amount of the cache group to which the LU 208 belongs is C, the cache monitor information Regarding the number of hits in the i-th area (0 <i <= R) in the total number of cache hits 370 in 362, the cache amount of the cache group to which the LU 208 belongs is reduced from iC / R to (i-1) C / R. The number of times that no hit occurs, and the number of hits in the i-th area (R <i <= R + V) is the number of hits when the cache amount is increased from (i-1) C / R to iC / R. it is conceivable that.
[0285]
Here, the average hit rate in the i-th area (0 <i <= R + V) at the time of executing the read access for each LU 208 is obtained by adding the actual area division number 502 in the storage device 40 to the monitor history information 510 shown in FIG. And the number of virtual management areas 504 are separately held in the system management information 142. For example, since the reduction of the average actual hit rate when the cache amount is reduced from C / R from C is the average hit ratio in the R-th area, the cache amount in the LU 208 is (i-1) from C / R to iC / R. (0 <i <= R), the average hit rate reduction per unit cache amount can be approximated as (average hit rate in the i-th area) × R / C. Similarly, when the cache amount is in the range of (i-1) C / R to iC / R (R <i <= R + V), the average hit rate increase per unit cache amount is (average hit rate in the i-th area) ) × R / C.
[0286]
This value is calculated for all LUs 208 that hold data used by Job, and is set as a hit rate change rate. In addition, it is assumed that a value based on the latest sample 524 or an average value of a predetermined period in the latest is acquired from the monitor history information 510 including the following steps.
[0287]
Next, using the Job resource mapping information in the system management information 412, the LU 208 storing the data used by Job and the HDD 16 corresponding thereto are grasped (Step 2704).
[0288]
Then, the average response time at the time of a read hit and the average response time at the time of a read miss of the HDD 16 are obtained by referring to the access performance information 614 of the HDD 16 of the HDD performance information 612, and (average response time at the time of read miss) − (average response time at the time of read hit). Average response time) is calculated and used as the response time change amount of the LU 208. When the data in the LU 208 is divided and stored in the HDDs 16 having different performances, the response time change amount is obtained for each HDD 16, and the value obtained by calculating the weighted average with the data amount stored in the HDD 16 is used as the response of the LU 208. Time change.
[0289]
For each storage device 40,
I = Σ (Average change in LU processing time)
However, (average change in LU processing time) = (change amount of response time) × (average number of read I / O times of LU) × (change amount of hit rate)
, A method of changing the cache allocation that maximizes I is obtained (step 2705).
[0290]
Here, Σ means that the sum of LUs 208 holding data of the specified Job in a certain storage device 40 is calculated. In addition, the response time change amount used in step 2704 is used, and the average read I / O count of the LU 208 is obtained from the monitor history information 510. The hit rate change amount can be obtained from the hit rate change rate obtained in step 2703 and the cache allocation change amount. At this time, note that the hit rate change rate changes depending on the cache amount of the cache group.
[0291]
For maximization, for example, the following algorithm is used.
[0292]
First, I is calculated for a certain cache group when the unit cache amount is increased or decreased. Then, when the cache amount is reduced, the cache allocation for the unit amount is changed from the cache group having the largest I (the effect is small) to the cache group having the largest value of the I (the effect is large) when the cache amount is increased. In this case, the value of I is recalculated, and when the value is equal to or more than a certain threshold value, the cache allocation is changed as “effective”. Hereinafter, the above confirmation is repeatedly performed assuming that the cache allocation has been changed, and it is determined that there is no effect in further performing the allocation change, or a predetermined amount (for example, when the Job is stored in the storage device 40, When the allocation is changed by a certain percentage of the used cache amount), the repetition of this check is terminated. If a cache group for which the cache allocation must not be reduced is specified at the start of the process, the restrictions are to be observed.
[0293]
Then, the process ends (step 2706). At this time, the value of the allocation information 652 in the Job cache amount information 650 in the Job resource management information 620 in FIG. 16 is updated.
[0294]
Next, a process of determining a method of changing the I / O processing performance will be described with reference to FIG.
FIG. 26 is a flowchart of a process for determining a method of changing the I / O processing performance.
[0295]
This process is a process of adjusting and changing the assignment of the job and the port, and is called from step 1303 of the process of executing the job in FIG. 22 and steps 1515 and 1517 of the job tuning process of FIGS. Processing.
[0296]
Also, in this process, when a new job is added from a higher-level routine at the start of the process or when the assignment is released when the job is completed, the identifier of the job and information indicating addition / release are given. To change the already allocated I / O processing amount, the identifier of the Job to be changed, the identifier of the port 26 whose allocation is to be changed, and the performance ratio 632 to be newly set to the port 26 are given (step 1601). . Unless otherwise specified, the I / O processing amount requested by another Job is grasped from the Job port setting information 630 and the Job management information 740 of the Job identified therefrom.
[0297]
First, the port 26 of the I / O path I / F 32 in the storage device 40 to be changed is identified (step 1602).
[0298]
When a job is newly added or deleted, the job resource mapping information is referenced from the job identifier, all the ports 26 used by the job are grasped, and these are set as the change targets of the processing amount allocation. When the I / O processing amount of the port 26 is directly specified, the processing amount allocation is changed only for the specified port 26. Thereafter, all currently executing Jobs using the obtained port 26 are grasped using the Job resource mapping information in the system management information 412.
[0299]
Next, in each port 26 obtained in step 1602, an allocation plan is created that allocates the I / O processing amount according to the request value of each job (step 1603).
[0300]
Here, the request value is a value when the performance ratio 632 is specified when the process is called, and otherwise, it is obtained as follows.
[0301]
First, for each data accessed by the Job through the port 26 obtained from the Job resource mapping information, the I / O processing content information 740 in the Job management information of the Job is referred to, and the value of the access order 742 corresponding to the data is referred to. Group by. Next, for each group, the sum of the values obtained by converting the performance value of the I / O base performance 748 of the data belonging to the port 26 into the performance ratio 632 is calculated. The calculated value in each group is compared with the current performance ratio 632 in the job port setting information 630, and the largest value is set as the request value.
[0302]
It is checked whether the allocation plan obtained in step 1603 is feasible, that is, whether the sum of the request values from each Job in each port 26 does not exceed 100% (step 1604).
[0303]
If there is an unrealizable port, the process proceeds to step 1605. If it can be realized by all ports, the process proceeds to step 1610, where the current allocation plan is used as the I / O processing amount allocation method, and the process ends.
[0304]
If it is determined in step 1604 that all the ports can be realized, the allocation plan is recalculated only for the port 26 which is determined to have a problem in step 1604.
[0305]
First, the reference time of the monitor information used for confirming the load of the job for online business is determined (step 1605).
[0306]
Basically, a time given in advance (for example, one hour) from the current time is set as the time. However, with respect to the Job obtained in step 1602, when a certain Job is managed by the batch Job management information 700, the process is executed for a time given by the value of the maximum execution time 710 from the time when the Job was started. When the end time is earlier than the end time of the above-described reference period, the estimated execution end time of the job may be used as the end time of the reference time of the monitor information.
[0307]
Next, an allocation plan is created in which the I / O processing amount is reduced within a range in which the required performance is not affected (step 1606).
[0308]
If the processing amount is specified when the processing is called, the assigned value is set to that value.
[0309]
For a Job managed by the batch Job management information 700, the calculation is performed as follows. First, it is assumed that the processing performance of the port 26 for which the assigned amount is not determined is a value obtained by multiplying the required value by a correction coefficient determined for each Job. At that time, the processing execution time estimated value is given by the maximum execution time 710. Is calculated. A simple equation for obtaining the correction coefficient is obtained by following the description of Job's processing execution time estimating method, and the value is obtained by solving the equation. Using the job priority 704 in the batch job management information 700 and referring to the priority control information 880, a margin coefficient 882 and a value obtained by multiplying the calculated correction coefficient by the required value are compared with the required value. Is the assigned value.
[0310]
The job managed by the online job management information 770 is requested as follows. First, the LU 208 that stores data accessed via the port 26 for which the assigned amount is not determined is obtained from the Job resource mapping information. The past I / O processing amount to the LU 208 via the target port 26 is grasped from the monitor history information 510. The data referred at this time are the monitor value within the reference time of the monitor information obtained in step 1605 within a predetermined period (for example, one week) and the monitor value immediately before. The request value is compared with a value obtained by multiplying the maximum value of the performance ratio 632 calculated from the monitor value by the margin coefficient 882 obtained from the job priority 704 in the online job management information 770, and the smaller one is set as the allocation value.
[0311]
Then, it is checked whether the sum of the obtained assignment values from each Job in each port 26 does not exceed 100% (step 1607). For the ports 26 whose sum does not exceed 100%, the assigned value is corrected so that the sum becomes 100%. This correction is performed by sequentially assigning the difference from the request value in order from the job with the highest job priority 704.
[0312]
If the problem that the sum of the assigned values exceeds 100% at one or more ports 26 is not solved, the process proceeds to step 1608. If the problem has been solved for all ports, the process proceeds to step 1610, where the current allocation plan is used as the I / O processing amount allocation method, and the process ends.
[0313]
Thereafter, a process of allocating an I / O processing amount to the port 26 in which the problem has not been solved up to step 1607 is performed in consideration of the job priority (step 1608).
[0314]
For example, for each port 26 for which an assignment has not been determined, the amount of each Job assigned to that port 26 is obtained by adding a new correction coefficient determined for each port 26 to the assigned amount obtained in step 1606 (Job priority given to Job). (A value of the degree 704), and a correction coefficient is determined so that the sum thereof becomes 100%, and the value is set as an assigned value to each Job.
[0315]
Regarding the Job accessing the data via the port 26 for which the I / O processing amount allocation plan is created in step 1608, the estimated processing time by the allocation plan is the maximum execution time in the Job managed by the batch Job management information 700. If the port 710 is not satisfied, the required performance may not be satisfied if there is a port 26 for which the allocation value obtained in step 1606 of the I / O processing amount cannot be secured in the Job managed by the online Job management information 770. Then, information indicating the high possibility is output (step 1609).
[0316]
This may be displayed on the screen 114 of the management terminal 110, or may be added to a log in which the system management program 140 stores the history of various events.
[0317]
Then, the process ends (step 1610).
[0318]
Next, a process of determining a method of eliminating an HDD overload state due to data movement based on monitor information will be described with reference to FIG.
FIG. 27 is a flowchart of a process for determining a method for eliminating an HDD overload state due to data movement based on monitor information.
[0319]
This process is a process called from step 1508 of the job tuning process of FIG.
[0320]
At the start of the process, the identifiers of the storage device 40 having the source HDD 16 as the source of data, the source HDD 16 and the LU 208 for transferring the data stored therein are given from the upper routine (step 2901).
[0321]
First, the latest state of the area mapping information 300 is acquired from each virtualization mechanism, and the mapping aggregation information is updated to the latest state. Thereafter, it is grasped as a “free area” based on the lower management structure corresponding area information 304 corresponding to the free entry 314 in the area mapping information 300 of each virtualization mechanism, and if possible, the virtualization switch 60 and the volume manager 78 Free the free area (Step 2902)
Here, the “free area” is used as a destination of data of a “data migration request” already existing in the system management information 142 in the lower management structure corresponding area information 304 corresponding to the free entry 314. Not something. The data movement request is information on a method for changing the correspondence between the LU 208 and the HDD 16 by data movement in a certain storage device 40 created by the processing started from step 2901. The information includes the device ID 572 of the HDD 16, the HDD ID 394 of the HDD 16 as the transfer source, the LU ID 364 of the LU 208 to which data is to be transferred, the HDD ID 394 of the HDD 16 as the transfer destination, and information on the area therein. This information is stored in the system management information 142.
[0322]
Of the grasped free areas, the free area in the virtualization switch 60 and the volume manager 78 can be released by issuing an instruction to release the area to the virtualization switch 60 and the volume manager 78 if possible. it can. When a certain area is released, what constitutes the opened area is a virtual structure, and if all the areas of the virtual structure are not used in the upper hierarchy, the virtual structure is deleted. The virtualization mechanism to be provided is instructed to delete the virtual structure, and the area of the management structure forming the virtual structure is made free. Hereinafter, this area release processing is performed recursively, and the free area is also grasped for them.
[0323]
Among the HDDs 16 in the storage device 40 having the source HDD 16, the HDD 16 capable of storing all the data of the designated LU 208 stored on the source HDD 16 in a continuous free area is determined, and the average of the HDD 16 at the time of a read error is determined. The response time is obtained from the access performance information 614 in the HDD performance information 612, and the HDD 16 whose value is equal to or less than the value of the source HDD 16 (read miss performance equal to or higher than the original HDD 16) is selected (step 2903). Note that the HDD 16 may not be selected.
[0324]
In the selected HDD 16, a sample 524 of the operation rate of the HDD 16 in a recent certain period (for example, one week) in the monitor history information 510 is grasped. If this happens, the operation rate of the entire HDD 16 after data movement (total of the operation rates of the HDD 16 in the LU 208 that stores data in the HDD 16) is estimated (step 2904).
[0325]
Here, the operation rate of the HDD 16 based on the data of the moved LU 208 is estimated as a value obtained by correcting the operation rate of the LU 208 in the source HDD 16 assuming that the operation rate of the HDD 16 at the destination and the source is proportional to the read miss performance.
[0326]
Of the estimated values of each sample 524, the largest value is taken as the operation rate after the data transfer of the HDD 16, and the HDD 16 with the smallest value is selected, and whether the value is less than a predetermined threshold value (for example, 50%) or not. Check whether or not. If it is less than the threshold value, a data movement request for moving the data of the LU 208 in the transfer source HDD 16 is created (step 2905). If the number is equal to or larger than the threshold value, or if no HDD 16 is selected in step 2903, the data migration request is not created as creation failure.
[0327]
Then, the process ends (step 2906).
[0328]
[Embodiment 2]
Hereinafter, a second embodiment according to the present invention will be described with reference to FIG.
FIG. 28 is a configuration diagram of a computer system that manages the performance of a storage device according to the second embodiment of the present invention.
[0329]
In the first embodiment, the virtualization switch 60 is placed between the server 70 and the storage device 40, and is connected to the I / O path 34 and the network 24. Then, the data is virtualized by a management structure as shown in FIG.
[0330]
In the present embodiment, the storage device 40b provides a file to an external device, and the file is accessed via a network using a network file system protocol.
[0331]
Hereinafter, the difference from the first embodiment will be described.
In the present embodiment, the I / O path 34 and the virtualization switch 60 do not exist. A port ID 384 that can be uniquely identified in the system is added to the port 26 of the network I / F 24.
[0332]
The server 70 does not have the I / O path I / F 32 but has only the network I / F 22. The OS 72 includes a network file system 82 that accesses a file 202 provided by an external device via the network 24 using a network file system protocol.
[0333]
The network file system 82 has the area mapping information 300 in the OS management information 74. When the file 202 recognized by the DBMS 90 or the AP program 100 and the file 202 provided from the storage device 40b correspond to each other according to a certain rule, only information that defines the correspondence may be held in the OS management information 74. . In this case, the system management program 140 acquires information that defines the correspondence, creates area mapping information 300 from the information, and stores it in the mapping aggregate information.
[0334]
The storage device 40b does not have the I / O path I / F 32 and has a concept called a "file" and allows an external device to access data. It is changed to the control program 44b of the storage device 40b and has the following functions. The control program 44b has a function provided by the file system 80 of the first embodiment, virtualizes the storage area of the LU 208 existing in the storage device 40b, and enables data access using the concept of a file 202. . Further, the control program 44b interprets one or more network file system protocols, and executes a file access process requested by an external device via the network 24 and the network I / F 22 using the protocols.
[0335]
Regarding the data mapping, in the data mapping hierarchy, all of the files 202 and below are provided by the storage device 40b, and the server 70 uses the network file system 82 in the OS 72 to store the files 202 on the storage device 40b. To access.
[0336]
The logical hierarchy in the storage device may include a logical volume 204 and a virtual volume 206 between the file 202 and the LU 208 as shown in FIG. 2, or may be simply a file 202-LU 208-HDD 16, A configuration may be used.
[0337]
Other differences are summarized below. The port 26 of the I / O path I / F 32 is made to correspond to the port 26 of the network I / F 22. This applies to the priority access control function of the storage device 40.
[0338]
The file 202 of the storage device 40b is made to correspond to the LU 208 of the storage device 40. The file 202 is also the unit of management of the members of the cache group.
[0339]
In the acquisition of the monitor information and various processes, the above-described changes are basically made, so that the first embodiment can be applied as it is.
[0340]
【The invention's effect】
According to the present invention, in a computer system in which a DBMS operates, performance management for a storage device can be performed using a performance index in a user's business, thereby facilitating performance management.
[0341]
Further, according to the present invention, in a computer system in which a DBSM operates, performance tuning processing of a storage device can be automated, and performance management costs can be reduced.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a computer system that manages the performance of a storage device according to a first embodiment of the present invention.
FIG. 2 is a hierarchical configuration diagram of data mapping of data managed by a DBMS 90 according to the first embodiment of the present invention.
FIG. 3 is a schematic diagram showing a data structure of area mapping information 300.
FIG. 4 is a schematic diagram showing a data structure of upper port mapping information 322.
FIG. 5 is a schematic diagram showing a data structure of data storage area information 342.
FIG. 6 is a schematic diagram showing a data structure of cache group information 460.
FIG. 7 is a schematic diagram showing a data structure of storage monitor information 360.
FIG. 8 is a schematic diagram showing a data structure of port throughput setting information 440.
FIG. 9 is a schematic diagram showing a data structure of port monitor information 400.
FIG. 10 is a schematic diagram showing a data structure of online Job monitor information 430.
FIG. 11 is a schematic diagram showing a data structure of storage performance information 610.
FIG. 12 is a schematic diagram showing a data structure of monitor history information 510.
FIG. 13 is a schematic diagram showing a data structure of batch Job management information 700.
FIG. 14 is a schematic diagram showing a data structure of batch SQL design information 850.
FIG. 15 is a schematic diagram showing a data structure of online Job management information 770.
FIG. 16 is a schematic diagram showing a data structure of Job resource management information 620.
FIG. 17 is a schematic diagram showing a data structure of priority control information 880.
FIG. 18 is a flowchart of a process for setting a CPU processing time estimation method 716 and I / O processing content information 740a in the job management information 700 by actually measuring monitor information.
FIG. 19 is a general chart of collection of monitor information of a system management program and Job tuning processing based on the monitor information.
FIG. 20 is a flowchart of a Job tuning process based on monitor information (part 1).
FIG. 21 is a flowchart of a Job tuning process based on monitor information (part 2).
FIG. 22 is a flowchart of Job execution processing by the system management program.
FIG. 23 is a flowchart of a process in which the system management program 140 determines allocation of a cache amount when starting Job.
FIG. 24 is a flowchart of a cache reallocation process executed by the system management program 140 as a job termination process.
FIG. 25 is a flowchart of a process of tuning a cache amount for online Job based on monitor information.
FIG. 26 is a flowchart of a process for determining a method of changing the I / O processing performance.
FIG. 27 is a flowchart of a process for determining a method for eliminating an HDD overload state due to data movement based on monitor information.
FIG. 28 is a configuration diagram of a computer system that manages the performance of a storage device according to the second embodiment of the present invention.
[Explanation of symbols]
16 HDD, 22 network I / F, 24 network, 26 port, 32 I / O path I / F, 34 I / O path, 40 storage device, 60 virtualization switch, 70 server , 90 ... DBMS, 100 ... AP program, 120 ... Management server, 140 ... System management program.

Claims (36)

In a performance management method for a computer system having a computer that executes a program and a storage device that stores data for executing the program,
(1) a procedure for acquiring performance information required for the program;
(2) a step of acquiring data mapping information for executing the program from the computer and the storage device;
(3) a procedure for acquiring operation statistical information from the storage device;
(10) a procedure for creating a setting change plan for the storage device using the information obtained in the procedures (1) to (3);
(11) A performance management method comprising a step of instructing the storage device to change a setting in accordance with the setting change plan. further,
(4) a step of obtaining information on the priority of the program,
2. The performance management method for a computer system according to claim 1, wherein, in the step (10), the setting change plan of the storage device is created using the information acquired in the step (4). The procedure (1) includes a process execution time required for the program, a process execution time estimating method, and a process of acquiring a data amount of data provided as input to the program,
In the step (10), when creating a setting change plan for the storage device, utilizing the processing execution time of the program, the processing execution time estimating method, and the amount of data input to the program. 2. The performance management method for a computer system according to claim 1, wherein: 4. The performance management method for a computer system according to claim 3, wherein the operation statistical information obtained in the step (3) is used when obtaining the processing execution time estimation method. The procedure of (1) includes a process of acquiring information on a response time of a process requested by the program and information on an actual measurement value of a response time of a process of the program,
In the step (10), when creating a setting change plan for the storage device, information on a response time of a process required for the program and information on an actual measurement value of a response time of a process of the program are stored. 2. The performance management method for a computer system according to claim 1, wherein the method is used. When the storage device is a storage device having a port for connecting to an external device and having a priority access control function of limiting the processing amount of access from the port,
2. The performance management method for a computer system according to claim 1, wherein the setting change plan created in the step (10) is a change plan of the setting value of the priority access control function. A function in which the storage device includes a data cache, manages the data cache by an area divided into several areas, and dynamically adds, deletes, and changes the area size of the divided data cache area. If the storage device has
2. The performance management method for a computer system according to claim 1, wherein the setting change plan created in the step (10) is a plan to add / delete / change the area size of the partitioned data cache. The storage device includes at least two or more physical storage units,
It has a function of providing a logical storage area using a storage area of the physical storage means, and a physical storage location change function of dynamically changing a correspondence relationship between the logical storage area and the storage area of the physical storage means. ,
2. The method according to claim 1, wherein the setting change plan created in the step (10) is a change instruction plan for changing the correspondence between the physical storage unit and the logical storage area by the physical storage location change function. The performance management method of the computer system described in the above. A first computer that operates a database management system, a second computer that executes a program that issues a processing request to the database management system, and a storage device that stores data handled by the database management system. In the performance management method,
(21) a procedure for acquiring performance information required for the program;
(22) a step of acquiring information on data of a database used in a process requested by the program to the database management system;
(23) a step of acquiring data mapping information for executing the program from the first computer and the storage device;
(24) a procedure for obtaining operation statistics information from the storage device;
(30) a procedure for creating a setting change plan for the storage device using the information acquired in the procedures (21) to (24);
(31) A method for managing the performance of a computer system, comprising a step of instructing the storage device to change a setting in accordance with the setting change plan. 10. The performance management method for a computer system according to claim 9, wherein said first computer and said second computer are the same computer. When the storage device is a storage device having a port for connecting to an external device and having a priority access control function of limiting the processing amount of access from the port,
10. The performance management method for a computer system according to claim 9, wherein the setting change plan created in the step (30) is a change plan of the setting value of the priority access control function. A function in which the storage device includes a data cache, manages the data cache by an area divided into several areas, and dynamically adds, deletes, and changes the area size of the divided data cache area. If the storage device has
10. The performance management method for a computer system according to claim 9, wherein the setting change plan created in the step (30) is a plan for adding / deleting / changing an area size of the partitioned data cache. The storage device includes at least two or more physical storage units,
A storage having a function of providing a logical storage area using a storage area of the physical storage means, and a physical storage location change function of dynamically changing a correspondence relationship between the logical storage area and the storage area of the physical storage means. If it is a device,
The method according to claim 30, wherein the setting change plan created in the step (30) is a change instruction plan for changing the correspondence between the physical storage unit and the logical storage area by the physical storage location change function. Item 14. The computer system performance management method according to Item 9. A first computer that operates a database management system, a second computer that executes a program that issues a processing request to the database management system, and a storage device that stores data handled by the database management system. In a computer system that manages performance,
Having a management device,
The management device may include information on performance required for the program,
Information on data of a database used in a process requested by the program to the database management system;
The first computer, mapping information of data for executing the program obtained from the storage device,
Acquiring operation statistics information obtained from the storage device,
The management device creates a setting change plan for the storage device using the acquired information,
A computer system for managing the performance of a storage device, which instructs the storage device to change the setting in accordance with the setting change plan. 15. The computer system according to claim 14, wherein the first computer and the second computer are the same computer. further,
15. The computer system for managing performance of a storage device according to claim 14, further comprising a storage control device connected between said first computer and said storage device for controlling data transfer between said first computer and said storage device. . The first computer or the second computer implements a function of collecting the information of the management device, creating the setting change plan, and performing a setting change instruction according to the setting change plan. 15. A computer system for managing the performance of a storage device according to claim 14, wherein: 15. The storage device according to claim 14, wherein a function of collecting information of the management device, creating the setting change plan, and instructing a setting change in accordance with the setting change plan is realized by the storage device. A computer system that manages the performance of storage devices. 17. The storage control device according to claim 16, wherein a function of collecting information of the management device, creating the setting change plan, and instructing a setting change in accordance with the setting change plan is realized by the storage control device. A computer system that manages the performance of storage devices. A first computer that operates a database management system; a second computer that executes a program that issues a processing request to the database management system; and a storage device that stores data handled by the database management system. In a management device that manages performance,
The management device may include information on performance required for the program,
Information on data of a database used in a process requested by the program to the database management system;
The first computer, mapping information of data for executing the program obtained from the storage device,
Acquiring operation statistics information obtained from the storage device,
Using the obtained information to create a setting change plan of the storage device,
A management device for managing the performance of a computer system, wherein the management device instructs the storage device to change a setting according to the setting change plan. 21. The management device for managing the performance of a computer system according to claim 20, wherein said first computer and said second computer are the same computer. The storage device has a port for connecting to an external device, a storage device having a priority access control function of limiting the processing amount of access from the port,
21. The management apparatus for managing the performance of a computer system according to claim 20, wherein the setting change plan created by the management apparatus is a change plan of a setting value of the priority access control function. A function in which the storage device includes a data cache, manages the data cache by an area divided into several areas, and dynamically adds, deletes, and changes the area size of the divided data cache area. A storage device having
21. The management apparatus for managing the performance of a computer system according to claim 20, wherein the setting change plan created by the management apparatus is a plan for adding / deleting / changing an area size of the partitioned data cache. The storage device includes at least two or more physical storage units,
A storage having a function of providing a logical storage area using a storage area of the physical storage means, and a physical storage location change function of dynamically changing a correspondence relationship between the logical storage area and the storage area of the physical storage means. A device,
21. The performance of a computer system according to claim 20, wherein the setting change plan created by the management device is a change instruction plan for changing the correspondence between the physical storage unit and the logical storage area by the physical storage location change function. Management device that manages In a performance management program for a computer system having a computer that executes a program and a storage device that stores data for executing the program,
A function of acquiring performance information required for the program,
A function of acquiring data mapping information for executing the program from the computer and the storage device;
With the function of acquiring operation statistical information from the storage device,
A function of collecting information from the computer system, creating a setting change plan of the storage device using the collected information, and instructing the storage device to perform a setting change according to the setting change plan is realized. A computer system performance management program characterized in that: further,
A function of acquiring information on the priority of the program,
26. The performance management program for a computer system according to claim 25, wherein the computer system also collects information on the priority of the program, and uses the information to create a setting change plan for the storage device. In the function of obtaining information on the performance required for the program, the processing execution time required for the program, the processing execution time estimation method, and the data amount of data given as input to the program are obtained,
26. The method according to claim 25, wherein a process execution time of the program, a process execution time estimating method, and an amount of data input to the program are used when creating a setting change plan of the storage device. Computer system performance management program. 28. The computer system performance management program according to claim 27, wherein when acquiring the processing execution time estimation method, the operation statistical information acquired by the function is used. The function of acquiring information on the performance required for the program includes acquiring information on a response time of a process requested by the program and information on an actual measurement value of a response time of a process of the program, and 26. The method according to claim 25, wherein, when creating the setting change plan, information on a response time of a process requested by the program and information on an actual measurement value of a response time of a process of the program are used. Computer system performance management program. A computer-readable storage medium for recording a performance management program for a computer system according to claim 25. A first computer that operates a database management system, a second computer that executes a program that issues a processing request to the database management system, and a storage device that stores data handled by the database management system, , A performance management program for a computer system having a storage controller connected between the first computer and the storage device and controlling data transfer between the two.
A function of acquiring performance information required for the program,
A function of acquiring information on data of a database used in a process that the program requests from the database management system,
A function of acquiring mapping information of data for executing the program from the first computer and the storage device;
With the function of acquiring operation statistical information from the storage device,
A function of collecting information from the computer system, creating a setting change plan of the storage device using the collected information, and instructing the storage device to perform a setting change according to the setting change plan is realized. A computer system performance management program characterized in that: 32. The computer system performance management program according to claim 31, wherein said first computer and said second computer are the same computer. The first computer or the second computer implements a function of collecting the information of the management device, creating the setting change plan, and performing a setting change instruction according to the setting change plan. 32. The computer system performance management program according to claim 31, wherein: 33. The storage device according to claim 31, wherein a function of collecting information of the management device, creating the setting change plan, and instructing a setting change in accordance with the setting change plan is realized by the storage device. Computer system performance management program. 32. The storage control device, wherein a function of collecting information of the management device, creating the setting change plan, and instructing a setting change in accordance with the setting change plan is realized by the storage control device. Computer system performance management program. 32. A computer readable storage medium for recording a computer system performance management program according to claim 31.

Images