US20150127858A1 - Dynamic virtual machine (VM) input-output per second (IOPS) control based on application IO profiling & VM IO usage trend analysis - Google Patents
Dynamic virtual machine (VM) input-output per second (IOPS) control based on application IO profiling & VM IO usage trend analysis Download PDFInfo
- Publication number
- US20150127858A1 US20150127858A1 US14/072,919 US201314072919A US2015127858A1 US 20150127858 A1 US20150127858 A1 US 20150127858A1 US 201314072919 A US201314072919 A US 201314072919A US 2015127858 A1 US2015127858 A1 US 2015127858A1
- Authority
- US
- United States
- Prior art keywords
- iops
- throughput
- value
- time interval
- virtual machine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0613—Improving I/O performance in relation to throughput
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45579—I/O management, e.g. providing access to device drivers or storage
Definitions
- the present invention relates to controlling input-output (IO) performance of virtual machines (VMs), more particularly, to systems and methods for dynamically controlling IO performance of virtual machines based on IO profiles and usage trends of the virtual machines.
- IO input-output
- FIG. 1A shows a typical virtualized environment 100 having multiple virtual machines 102 a - 102 n.
- the virtual machines (VMs) 102 a - 102 n are installed on multiple VM host servers (not shown in FIG. 1 for brevity), and access a datastore 120 via the VM host servers.
- the customer of the virtual environment would like to allocate greater IO access to the high-priority virtual machines 102 a and 102 b than the low-priority virtual machine 102 n, such as a backup server.
- the high-priority virtual machines 102 a - 102 b may include IO sensitive and latency-sensitive applications, such as online stores and Exchange® server, for instance.
- the thicknesses of the arrows 108 a - 108 n indicate the intended amounts of IO access for the virtual machines 102 a - 102 n , respectively, in an ideal operational condition.
- FIG. 1B depicts an operational condition where the performance of the high-priority virtual machines 102 a and 102 b are adversely affected by the low-priority virtual machine 102 n.
- the low-priority virtual machine 102 n gets a significant portion of access to the datastore 120 while the high-priority virtual machines 102 and 102 b are not getting enough throughput for proper operations thereof.
- a computer-implemented method for controlling input-output (IO) requests of a plurality of virtual machines to a datastore includes: monitoring, for each virtual machine, a throughput of IO to a datastore for a preset time interval; identifying a peak value of the throughput; calculating a value of input-output-per-second (IOPS) using the peak value; and setting the value of IOPS as an IOPS limit for a corresponding virtual machine.
- IOPS input-output-per-second
- a computer readable medium or media comprising a set of instructions for controlling input-output (IO) requests of a plurality of virtual machines to a datastore, wherein execution of the set of instructions by one or more processors causes the one or more processors to perform the steps of: monitoring, for each virtual machine, a throughput of IO to a datastore for a preset time interval; identifying a peak value of the throughput; calculating a value of IOPS using the peak value; and setting the value of IOPS as an IOPS limit for a corresponding virtual machine.
- IO input-output
- a system for controlling IO requests of a virtual machine to a datastore includes at least one server that is adapted to monitor a throughput of IO to a datastore for a preset time interval, identify a peak value of the throughput, calculate a value of IOPS using the peak value, and using the value of IOPS to set an IOPS limit for the virtual machine.
- FIG. 1A shows a conventional virtualized environment under an ideal operational condition.
- FIG. 1B shows the virtualized environment of FIG. 1A under an actual operational condition where the performance of the virtual machines are significantly degraded by improper IO control.
- FIG. 2 shows an exemplary virtualized environment according to embodiments of the present invention.
- FIG. 3 shows a plot of throughput as a function of the number of IOs in flight for a virtual machine in FIG. 2 according to embodiments of the present invention.
- FIG. 4 shows exemplary IO profiles attached to the virtual machines of FIG. 2 according to embodiments of the present invention.
- FIG. 5 shows a plot of throughput saturated due to the underestimated IOPS limit according to embodiments of the present invention.
- FIG. 6 shows a flowchart of an illustrative process for controlling IO requests of multiple virtual machines to a datastore according to embodiments of the present invention.
- FIG. 7 shows a computer system according to embodiments of the present invention.
- connections between components within the figures are not intended to be limited to direct connections. Rather, data between these components may be modified, re-formatted, or otherwise changed by intermediary components. Also, additional or fewer connections may be used. It shall also be noted that the terms “coupled” or “communicatively coupled” shall be understood to include direct connections, indirect connections through one or more intermediary devices, and wireless connections.
- a service, function, or resource is not limited to a single service, function, or resource; usage of these terms may refer to a grouping of related services, functions, or resources, which may be distributed or aggregated.
- FIG. 2 shows an exemplary virtualized environment 200 according to embodiments of the present invention.
- the virtualized environment 200 includes a datastore 202 that is coupled to and shared by multiple computers 204 a - 204 n.
- Each of the computers 204 a - 204 n provides a physical computing platform to a corresponding one of VM host servers 208 a - 208 n.
- Each of the VM host servers 208 a - 208 n which are also known as hypervisors, hosts multiple VMs.
- the VM host server 208 a may host one or more VMs 212 a - 212 n and one or more servers 213 a - 213 n.
- the server 213 a may be a backup server of low-priority, for instance, and the server 213 n may be a centralized management application, such as (by way of illustration and not limitation) vCenter® server, by VMware of Palo Alto, California, that allows the system administrator to manage virtual machines 212 a - 212 n and the servers 213 a - 213 n.
- the server 213 n may be coupled to a client application 220 , such as (by way of illustration and not limitation) vSphere® by VMware of Palo Alto, Calif., client application, that is used by the system administrator to access the server 213 n and ultimately manage the VM host server 208 a.
- the VM host server 208 n may host one or more VMs 214 a - 214 n and one or more servers 215 a - 215 n, where the server 215 n may be similar to the server 213 n, i.e., the server 215 n may be a centralized management application that performs the similar functions as the server 213 n and coupled to the client application 220 .
- Each of the VMs 212 a - 212 n and 214 a - 214 n may include an operating system that executes an application, such as online store, for instance.
- an application such as online store
- VMs collectively refers to the VMs 212 a - 212 n and servers 213 a - 213 n.
- one VM host server may affect the performance of VMs hosted by other VM host servers. For instance, if the VM host server 208 a were to issue a large number of IO requests on behalf of its backup server 213 a so that the backup server 213 a could write enormous amount of data, the other VMs 214 a - 214 n, which may be IO intensive or latency-sensitive applications, would not have any access to the datastore 202 , significantly degrading the performance of the VMs 214 a - 214 n.
- the present invention prevents such an undesirable allocation of IO access to the datastore 202 by dynamic storage IO control.
- dynamic storage IO control is performed by use of IO profile, where a server, say 213 n, attaches an IO profile to each VM installed in the VM host server 208 a.
- the system administrator uses a client application 220 coupled to the server 213 n to attach an IO profile to the VM 212 a.
- the IO profile includes one or more parameters, such as IO block size.
- IO block size refers to a fixed length of a block, which is a sequence of bytes or bits and used to facilitate the handling of data-stream by a VM receiving or sending data from the datastore 202 .
- the IO block size may be set based on standard business critical applications. In one embodiment, a default IO block size may be assigned to each VM. In another embodiment, the application executed by a VM may have its own IO block size. In yet another embodiment, if the IO block size of a VM is not known, the system administrator may assign a proper IO block size inferred from other VM that executes a similar application.
- FIG. 3 shows a plot 300 of throughput as a function of the number of IOs in flight for a VM, say 212 a.
- the throughput refers to disk IO in the unit of MB/sec but other metric may be used.
- the throughput has a peak value at a point 302 , beyond which the throughput decreases as the number of IOs in flight increases.
- IOPS input-output-per-second
- IO block size is in the unit of kilobytes.
- the number of IOPS calculated by Equation (1) is an IOPS limit applied to the VM 212 a in the subsequent time interval, to thereby prevent the VM 212 a from utilizing more datastore resource than is required for its optimum operation.
- the server 213 n monitors the other VMs hosted by the VM host server 208 a and sets an IOPS limit to each VM. Also, the server 213 n continuously repeats the step of monitoring the throughput of each VM, identifying the peak point 302 and setting a new IOPS limit at each preset time interval, to thereby dynamically control the storage IO without further human intervention.
- a different value than the peak throughput may be used.
- a value within a set region or threshold level may be used.
- Equation 1 may be modified to be based upon 90% of the peak throughput.
- the plot 300 in FIG. 3 may be used to help a system administrator understand each VM's IO trend.
- the system administrator may manually set or adjust the IOPS limits to either relax or increase the IOPS limits, even though the servers 213 n and 215 n can automatically set the IOPS limits at the end of each time interval. For instance, the system administrator may analyze the IO trend of a VM at the end of each month and adjust the IOPS limit of the VM based on the trend analysis.
- FIG. 4 shows exemplary IO profiles 404 a - 404 m attached to the virtual machines 212 a - 214 n of FIG. 2 according to embodiments of the present invention.
- the IO profile of each VM includes IO block size and the IOPS limit calculated by Equation (1) at a time interval.
- the IO request slots 402 are apportioned and allocated to each VM in proportion to the IOPS limits of the VMs. As such, the number of IO requests to the data storage array 404 for one VM may be affected by the number of IO requests by other VMs.
- an IOPS limit for a VM in the subsequent time interval is calculated by monitoring the peak throughput of the VM in the current time interval.
- the IOPS limit calculated by Equation (1) is based on the assumption that the peak throughput in the subsequent time interval is the same as or less than the peak throughput in the current time interval. However, if the actual peak throughput in the subsequent time interval is higher than that of the current time interval, the throughput monitored in the subsequent time interval will be saturated.
- FIG. 5 shows a plot 500 of throughput saturated due to the underestimated peak throughput (i.e., underestimated IOPS limit) set in the previous time interval. As depicted, the actual throughput may have a peak point 502 while the monitored throughput is flattened out by the ceiling of the underestimated peak throughput 504 , where the underestimated peak throughput is the peak value of the throughput monitored in the previous time interval.
- the server 213 n may allow the system administrator to specify a tolerance so that the IOPS limit calculated by Equation (1) is adjusted according to the tolerance.
- the tolerance may be a factor to be multiplied to the calculated IOPS limit or a certain number of IOPS to be added to the calculated IOPS limits so that the throughput corresponding to the adjusted IOPS limit is higher than the actual peak throughput in the subsequent time interval.
- the system administrator may manually adjust the IOPS limit based on the trend plot of FIG. 3 or other factors such as expected IO loads, importance of data, load factors, etc.
- FIG. 6 shows a flowchart of an illustrative process 600 for controlling IO requests of multiple virtual machines to a datastore according to embodiments of the present invention.
- the process starts at step 602 .
- a system administrator attaches an IO profile to each VM, where the IO profile includes an IO block size.
- a default IO block size may be assigned to each VM.
- the application executed by a VM may have its own IO block size.
- the system administrator may assign a proper IO block size inferred from other VM that executes a similar application.
- the system administrator may access a centralized management server through the client server 220 .
- the system administrator may access the server 213 n to attach an IO profile to the VM 212 a.
- the server 213 n monitors, for each VM, a throughput to the datastore 202 during a preset time interval, such as 7 days (although other time intervals may be used).
- the monitored throughput of each VM may be plotted as a function of the number of IOs in flight.
- the server 213 n selects one VM amongst the VMs 212 a - 212 n and identifies the peak value of the throughput of the selected VM at steps 606 and 608 , respectively. Subsequently, the server 213 n calculates the number of IOPS at the peak value of the throughput. In embodiments, Equation (1) may be employed at step 610 to calculate the number of IOPS. Then, the calculated IOPS is set as a new IOPS limit for the selected VM in a subsequent time interval, such as 7 days, at step 612 .
- the IOPS limit calculated at step 612 is based on the assumption that the peak throughput in the subsequent time interval is the same as the peak throughput in the current time interval. If the actual peak throughput in the subsequent time interval is higher than that of the current time interval, the throughput monitored in the subsequent time interval will be saturated. To prevent the saturation, in embodiments, the server 213 n may include a tolerance so that the IOPS limit calculated by Equation (1) is adjusted according to the tolerance at the optional step 614 .
- the tolerance may be a factor to be multiplied to the calculated IOPS limit (i.e., a constant may be multiplied to the right hand side of Equation (1)) or a certain number of IOPS to be added to the calculated IOPS limit so that the throughput corresponding to the adjusted IOPS is higher than the actual peak throughput in the subsequent time interval.
- the system administrator may manually adjust the IOPS limit based on the analysis of the trend plot of FIG. 3 . Then, the process proceeds to step 616 .
- the server 213 n determines whether there is any other VM that needs a new IOPS limit. If the answer to the step 616 is positive, the process proceeds to step 606 so that steps 606 - 614 are repeated until every VM has a new IOPS limit. Otherwise, the process proceeds to step 618 .
- the IOPS limit(s) have been set and the monitoring may be stopped at step 620 .
- the IOPS limit(s) may be dynamically monitored on a continuous basis. For example, the subsequent time interval may be set as the current time interval and the process proceeds to step 604 to repeat steps 604 - 616 .
- the server 213 n monitors the throughputs of VMs hosted by the VM host server 208 a and sets new IOPS limits for the VMs at each time interval, i.e., the server 213 n continuously and dynamically controls the storage IO without further human intervention.
- the monitored throughput for each VM helps system administrator understand the throughput trend in a preset time interval to thereby assist him to manage the usage of datastore resources in the subsequent time interval.
- one or more computing system may be configured to perform one or more of the methods, functions, and/or operations presented herein.
- Systems that implement at least one or more of the methods, functions, and/or operations described herein may comprise an application or applications operating on at least one computing system.
- the computing system may comprise one or more computers and one or more databases.
- the computer system may be a single system, a distributed system, a cloud-based computer system, or a combination thereof.
- the present invention may be implemented in any instruction-execution/computing device or system capable of processing data, including, without limitation phones, laptop computers, desktop computers, and servers.
- the present invention may also be implemented into other computing devices and systems.
- aspects of the present invention may be implemented in a wide variety of ways including software (including firmware), hardware, or combinations thereof.
- the functions to practice various aspects of the present invention may be performed by components that are implemented in a wide variety of ways including discrete logic components, one or more application specific integrated circuits (ASICs), and/or program-controlled processors. It shall be noted that the manner in which these items are implemented is not critical to the present invention.
- system 700 includes a central processing unit (CPU) 701 that provides computing resources and controls the computer.
- CPU 701 may be implemented with a microprocessor or the like, and may also include a graphics processor and/or a floating point coprocessor for mathematical computations.
- System 700 may also include a system memory 702 , which may be in the form of random-access memory (RAM) and read-only memory (ROM).
- RAM random-access memory
- ROM read-only memory
- An input controller 703 represents an interface to various input device(s) 704 , such as a keyboard, mouse, or stylus.
- a scanner controller 705 which communicates with a scanner 706 .
- System 700 may also include a storage controller 707 for interfacing with one or more storage devices 708 each of which includes a storage medium such as magnetic tape or disk, or an optical medium that might be used to record programs of instructions for operating systems, utilities and applications which may include embodiments of programs that implement various aspects of the present invention.
- Storage device(s) 708 may also be used to store processed data or data to be processed in accordance with the invention.
- System 700 may also include a display controller 709 for providing an interface to a display device 711 , which may be a cathode ray tube (CRT), a thin film transistor (TFT) display, or other type of display.
- System 700 may also include a printer controller 712 for communicating with a printer 713 .
- a communications controller 714 may interface with one or more communication devices 715 , which enables system 700 to connect to remote devices through any of a variety of networks including the Internet, a local area network (LAN), a wide area network (WAN), or through any suitable electromagnetic carrier signals including infrared signals.
- LAN local area network
- WAN wide area network
- electromagnetic carrier signals including infrared signals.
- bus 716 which may represent more than one physical bus.
- various system components may or may not be in physical proximity to one another.
- input data and/or output data may be remotely transmitted from one physical location to another.
- programs that implement various aspects of this invention may be accessed from a remote location (e.g., a server) over a network.
- Such data and/or programs may be conveyed through any of a variety of machine-readable medium including, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices that are specially configured to store or to store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices.
- ASICs application specific integrated circuits
- PLDs programmable logic devices
- flash memory devices ROM and RAM devices.
- Embodiments of the present invention may be encoded upon one or more non-transitory computer-readable media with instructions for one or more processors or processing units to cause steps to be performed.
- the one or more non-transitory computer-readable media shall include volatile and non-volatile memory.
- alternative implementations are possible, including a hardware implementation or a software/hardware implementation.
- Hardware-implemented functions may be realized using ASIC(s), programmable arrays, digital signal processing circuitry, or the like. Accordingly, the “means” terms in any claims are intended to cover both software and hardware implementations.
- the term “computer-readable medium or media” as used herein includes software and/or hardware having a program of instructions embodied thereon, or a combination thereof.
- embodiments of the present invention may further relate to computer products with a non-transitory, tangible computer-readable medium that have computer code thereon for performing various computer-implemented operations.
- the media and computer code may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind known or available to those having skill in the relevant arts.
- Examples of tangible computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices that are specially configured to store or to store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices.
- ASICs application specific integrated circuits
- PLDs programmable logic devices
- flash memory devices and ROM and RAM devices.
- Examples of computer code include machine code, such as produced by a compiler, and files containing higher level code that are executed by a computer using an interpreter.
- Embodiments of the present invention may be implemented in whole or in part as machine-executable instructions that may be in program modules that are executed by a processing device.
- Examples of program modules include libraries, programs, routines, objects, components, and data structures. In distributed computing environments, program modules may be physically located in settings that are local, remote, or both.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Debugging And Monitoring (AREA)
Abstract
Description
- The present invention relates to controlling input-output (IO) performance of virtual machines (VMs), more particularly, to systems and methods for dynamically controlling IO performance of virtual machines based on IO profiles and usage trends of the virtual machines.
- Controlling the dynamic allocation of resources in distributed systems has been a long-standing challenge. Virtualized environments introduce further challenges because of the inherent sharing of physical resources by multiple virtual machines. Existing software applications have provided ways to manage shared physical resources, such as CPU and memory, for the virtualized environments and to prioritize their use among all the virtual machines in the virtualized environments. CPU and memory controls have worked well since the memory and CPU resources are shared only at a local-host level for virtual machines residing within a single VM host server.
- However, the task of regulating shared resources that span multiple VM host servers, such as shared datastores, presents new challenges since these resources are accessed in a distributed manner by multiple VM host servers. Conventional approaches to share disk storage do not address this challenge, as the shares and limits were enforced only at a single VM host server level, and were only enforced in response to host-side host-bus-adaptor (HBA) bottlenecks, which occur rarely. These approaches have the problem of potentially allowing lower-priority virtual machines to have greater access to storage resources based on their placement across different VM host severs.
FIG. 1A shows a typicalvirtualized environment 100 having multiple virtual machines 102 a-102 n. The virtual machines (VMs) 102 a-102 n are installed on multiple VM host servers (not shown inFIG. 1 for brevity), and access adatastore 120 via the VM host servers. The customer of the virtual environment would like to allocate greater IO access to the high-priority 102 a and 102 b than the low-priorityvirtual machines virtual machine 102 n, such as a backup server. The high-priority virtual machines 102 a-102 b may include IO sensitive and latency-sensitive applications, such as online stores and Exchange® server, for instance. As depicted inFIG. 1A , the thicknesses of the arrows 108 a-108 n indicate the intended amounts of IO access for the virtual machines 102 a-102 n, respectively, in an ideal operational condition. - However, the conventional disk share approaches allocate storage resources based on the placement of virtual machines across multiple VM host servers. Under such existing approaches, the low-priority
virtual machine 102 n may write enormous amount of data even though it is not necessary to guarantee thevirtual machine 102 n for that level of IO throughput.FIG. 1B depicts an operational condition where the performance of the high-priority 102 a and 102 b are adversely affected by the low-priorityvirtual machines virtual machine 102 n. As depicted by arrows 110 a-110 n, the low-priorityvirtual machine 102 n gets a significant portion of access to thedatastore 120 while the high-priorityvirtual machines 102 and 102 b are not getting enough throughput for proper operations thereof. - As such, there is a need for systems and methods for controlling IO performance of virtual machines that share common datastore resources.
- In embodiments, a computer-implemented method for controlling input-output (IO) requests of a plurality of virtual machines to a datastore includes: monitoring, for each virtual machine, a throughput of IO to a datastore for a preset time interval; identifying a peak value of the throughput; calculating a value of input-output-per-second (IOPS) using the peak value; and setting the value of IOPS as an IOPS limit for a corresponding virtual machine.
- In embodiments, a computer readable medium or media comprising a set of instructions for controlling input-output (IO) requests of a plurality of virtual machines to a datastore, wherein execution of the set of instructions by one or more processors causes the one or more processors to perform the steps of: monitoring, for each virtual machine, a throughput of IO to a datastore for a preset time interval; identifying a peak value of the throughput; calculating a value of IOPS using the peak value; and setting the value of IOPS as an IOPS limit for a corresponding virtual machine.
- In embodiments, a system for controlling IO requests of a virtual machine to a datastore includes at least one server that is adapted to monitor a throughput of IO to a datastore for a preset time interval, identify a peak value of the throughput, calculate a value of IOPS using the peak value, and using the value of IOPS to set an IOPS limit for the virtual machine.
- Some features and advantages of the invention have been generally described in this summary section; however, additional features, advantages, and embodiments are presented herein or will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims hereof. Accordingly, it should be understood that the scope of the invention shall not be limited by the particular embodiments disclosed in this summary section.
- References will be made to embodiments of the invention, examples of which may be illustrated in the accompanying figures. These figures are intended to be illustrative, not limiting. Although the invention is generally described in the context of these embodiments, it should be understood that it is not intended to limit the scope of the invention to these particular embodiments.
-
FIG. 1A shows a conventional virtualized environment under an ideal operational condition. -
FIG. 1B shows the virtualized environment ofFIG. 1A under an actual operational condition where the performance of the virtual machines are significantly degraded by improper IO control. -
FIG. 2 shows an exemplary virtualized environment according to embodiments of the present invention. -
FIG. 3 shows a plot of throughput as a function of the number of IOs in flight for a virtual machine inFIG. 2 according to embodiments of the present invention. -
FIG. 4 shows exemplary IO profiles attached to the virtual machines ofFIG. 2 according to embodiments of the present invention. -
FIG. 5 shows a plot of throughput saturated due to the underestimated IOPS limit according to embodiments of the present invention. -
FIG. 6 shows a flowchart of an illustrative process for controlling IO requests of multiple virtual machines to a datastore according to embodiments of the present invention. -
FIG. 7 shows a computer system according to embodiments of the present invention. - In the following description, for purposes of explanation, specific details are set forth in order to provide an understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these details. Furthermore, one skilled in the art will recognize that embodiments of the present invention, described below, may be implemented in a variety of ways, such as a process, an apparatus, a system, a device, or a method on a tangible computer-readable medium.
- Also, it shall be noted that steps or operations may be performed in different orders or concurrently, as will be apparent to one of skill in the art. And, in instances, well known process operations have not been described in detail to avoid unnecessarily obscuring the present invention.
- Components, or modules, shown in diagrams are illustrative of exemplary embodiments of the invention and are meant to avoid obscuring the invention. It shall also be understood that throughout this discussion that components may be described as separate functional units, which may comprise sub-units, but those skilled in the art will recognize that various components, or portions thereof, may be divided into separate components or may be integrated together, including integrated within a single system or component. It should be noted that functions or operations discussed herein may be implemented as components or modules. Components or modules may be implemented in software, hardware, or a combination thereof.
- Furthermore, connections between components within the figures are not intended to be limited to direct connections. Rather, data between these components may be modified, re-formatted, or otherwise changed by intermediary components. Also, additional or fewer connections may be used. It shall also be noted that the terms “coupled” or “communicatively coupled” shall be understood to include direct connections, indirect connections through one or more intermediary devices, and wireless connections.
- Reference in the specification to “one embodiment,” “preferred embodiment,” “an embodiment,” or “embodiments” means that a particular feature, structure, characteristic, or function described in connection with the embodiment is included in at least one embodiment of the invention and may be in more than one embodiment. The appearances of the phrases “in one embodiment,” “in an embodiment,” or “in embodiments” in various places in the specification are not necessarily all referring to the same embodiment or embodiments.
- The use of certain terms in various places in the specification is for illustration and should not be construed as limiting. A service, function, or resource is not limited to a single service, function, or resource; usage of these terms may refer to a grouping of related services, functions, or resources, which may be distributed or aggregated.
-
FIG. 2 shows an exemplary virtualizedenvironment 200 according to embodiments of the present invention. As depicted, thevirtualized environment 200 includes adatastore 202 that is coupled to and shared by multiple computers 204 a-204 n. Each of the computers 204 a-204 n provides a physical computing platform to a corresponding one of VM host servers 208 a-208 n. Each of the VM host servers 208 a-208 n, which are also known as hypervisors, hosts multiple VMs. For instance, theVM host server 208 a may host one or more VMs 212 a-212 n and one or more servers 213 a-213 n. Theserver 213 a may be a backup server of low-priority, for instance, and theserver 213 n may be a centralized management application, such as (by way of illustration and not limitation) vCenter® server, by VMware of Palo Alto, California, that allows the system administrator to manage virtual machines 212 a-212 n and the servers 213 a-213 n. In one embodiment, theserver 213 n may be coupled to aclient application 220, such as (by way of illustration and not limitation) vSphere® by VMware of Palo Alto, Calif., client application, that is used by the system administrator to access theserver 213 n and ultimately manage theVM host server 208 a. TheVM host server 208 n may host one or more VMs 214 a-214 n and one or more servers 215 a-215 n, where theserver 215 n may be similar to theserver 213 n, i.e., theserver 215 n may be a centralized management application that performs the similar functions as theserver 213 n and coupled to theclient application 220. - Each of the VMs 212 a-212 n and 214 a-214 n may include an operating system that executes an application, such as online store, for instance. Hereinafter, the term VMs collectively refers to the VMs 212 a-212 n and servers 213 a-213 n.
- Since the VM host servers 208 a-208 n share the
datastore 202, one VM host server may affect the performance of VMs hosted by other VM host servers. For instance, if theVM host server 208 a were to issue a large number of IO requests on behalf of itsbackup server 213 a so that thebackup server 213 a could write enormous amount of data, the other VMs 214 a-214 n, which may be IO intensive or latency-sensitive applications, would not have any access to thedatastore 202, significantly degrading the performance of the VMs 214 a-214 n. The present invention prevents such an undesirable allocation of IO access to thedatastore 202 by dynamic storage IO control. - In the present invention, dynamic storage IO control is performed by use of IO profile, where a server, say 213 n, attaches an IO profile to each VM installed in the
VM host server 208 a. In one embodiment, the system administrator uses aclient application 220 coupled to theserver 213 n to attach an IO profile to theVM 212 a. The IO profile includes one or more parameters, such as IO block size. - IO block size refers to a fixed length of a block, which is a sequence of bytes or bits and used to facilitate the handling of data-stream by a VM receiving or sending data from the
datastore 202. The IO block size may be set based on standard business critical applications. In one embodiment, a default IO block size may be assigned to each VM. In another embodiment, the application executed by a VM may have its own IO block size. In yet another embodiment, if the IO block size of a VM is not known, the system administrator may assign a proper IO block size inferred from other VM that executes a similar application. - After attaching an IO profile to a VM, the
server 213 n continuously monitor the VM's throughput for a preset time interval, such as 7 days (although other time intervals or even varying time intervals may be used), using a software tool, such as Vscsistats® software or Esxtop® software.FIG. 3 shows aplot 300 of throughput as a function of the number of IOs in flight for a VM, say 212 a. Here, the throughput refers to disk IO in the unit of MB/sec but other metric may be used. As depicted in thetrend plot 300, the throughput has a peak value at apoint 302, beyond which the throughput decreases as the number of IOs in flight increases. Thus, there is no benefit of allocating higher input-output-per-second (IOPS) than calculated at thepeak point 302. The number of IOPS at thepeak point 302 can be calculated by an equation -
Number of IOPS=(1024/IO block size)*peak throughput, (1) - where the IO block size is in the unit of kilobytes.
- In embodiments, the number of IOPS calculated by Equation (1) is an IOPS limit applied to the
VM 212 a in the subsequent time interval, to thereby prevent theVM 212 a from utilizing more datastore resource than is required for its optimum operation. Theserver 213 n monitors the other VMs hosted by theVM host server 208 a and sets an IOPS limit to each VM. Also, theserver 213 n continuously repeats the step of monitoring the throughput of each VM, identifying thepeak point 302 and setting a new IOPS limit at each preset time interval, to thereby dynamically control the storage IO without further human intervention. - In embodiments, a different value than the peak throughput may be used. For example, a value within a set region or threshold level may be used. For example, Equation 1 may be modified to be based upon 90% of the peak throughput.
- In embodiments, the
plot 300 inFIG. 3 may be used to help a system administrator understand each VM's IO trend. In embodiments, based on the trend analysis, the system administrator may manually set or adjust the IOPS limits to either relax or increase the IOPS limits, even though the 213 n and 215 n can automatically set the IOPS limits at the end of each time interval. For instance, the system administrator may analyze the IO trend of a VM at the end of each month and adjust the IOPS limit of the VM based on the trend analysis.servers -
FIG. 4 showsexemplary IO profiles 404 a-404 m attached to the virtual machines 212 a-214 n ofFIG. 2 according to embodiments of the present invention. As depicted, the IO profile of each VM includes IO block size and the IOPS limit calculated by Equation (1) at a time interval. TheIO request slots 402 are apportioned and allocated to each VM in proportion to the IOPS limits of the VMs. As such, the number of IO requests to thedata storage array 404 for one VM may be affected by the number of IO requests by other VMs. - As discussed in conjunction with
FIG. 3 , an IOPS limit for a VM in the subsequent time interval is calculated by monitoring the peak throughput of the VM in the current time interval. Thus, the IOPS limit calculated by Equation (1) is based on the assumption that the peak throughput in the subsequent time interval is the same as or less than the peak throughput in the current time interval. However, if the actual peak throughput in the subsequent time interval is higher than that of the current time interval, the throughput monitored in the subsequent time interval will be saturated.FIG. 5 shows aplot 500 of throughput saturated due to the underestimated peak throughput (i.e., underestimated IOPS limit) set in the previous time interval. As depicted, the actual throughput may have apeak point 502 while the monitored throughput is flattened out by the ceiling of the underestimatedpeak throughput 504, where the underestimated peak throughput is the peak value of the throughput monitored in the previous time interval. - To prevent the saturation, in embodiments, the
server 213 n may allow the system administrator to specify a tolerance so that the IOPS limit calculated by Equation (1) is adjusted according to the tolerance. The tolerance may be a factor to be multiplied to the calculated IOPS limit or a certain number of IOPS to be added to the calculated IOPS limits so that the throughput corresponding to the adjusted IOPS limit is higher than the actual peak throughput in the subsequent time interval. Alternatively, the system administrator may manually adjust the IOPS limit based on the trend plot ofFIG. 3 or other factors such as expected IO loads, importance of data, load factors, etc. -
FIG. 6 shows a flowchart of anillustrative process 600 for controlling IO requests of multiple virtual machines to a datastore according to embodiments of the present invention. In embodiments, the process starts atstep 602. Atstep 602, a system administrator attaches an IO profile to each VM, where the IO profile includes an IO block size. In embodiments, a default IO block size may be assigned to each VM. In other embodiments, the application executed by a VM may have its own IO block size. In yet other embodiments, if the IO block size of a VM is not known, the system administrator may assign a proper IO block size inferred from other VM that executes a similar application. - To attach an IO profile to a VM, the system administrator may access a centralized management server through the
client server 220. For instance, the system administrator may access theserver 213 n to attach an IO profile to theVM 212 a. Then, atstep 604, theserver 213 n monitors, for each VM, a throughput to thedatastore 202 during a preset time interval, such as 7 days (although other time intervals may be used). In one embodiment, the monitored throughput of each VM may be plotted as a function of the number of IOs in flight. Theserver 213 n selects one VM amongst the VMs 212 a-212 n and identifies the peak value of the throughput of the selected VM at 606 and 608, respectively. Subsequently, thesteps server 213 n calculates the number of IOPS at the peak value of the throughput. In embodiments, Equation (1) may be employed atstep 610 to calculate the number of IOPS. Then, the calculated IOPS is set as a new IOPS limit for the selected VM in a subsequent time interval, such as 7 days, atstep 612. - In embodiments, the IOPS limit calculated at
step 612 is based on the assumption that the peak throughput in the subsequent time interval is the same as the peak throughput in the current time interval. If the actual peak throughput in the subsequent time interval is higher than that of the current time interval, the throughput monitored in the subsequent time interval will be saturated. To prevent the saturation, in embodiments, theserver 213 n may include a tolerance so that the IOPS limit calculated by Equation (1) is adjusted according to the tolerance at theoptional step 614. The tolerance may be a factor to be multiplied to the calculated IOPS limit (i.e., a constant may be multiplied to the right hand side of Equation (1)) or a certain number of IOPS to be added to the calculated IOPS limit so that the throughput corresponding to the adjusted IOPS is higher than the actual peak throughput in the subsequent time interval. Alternatively, the system administrator may manually adjust the IOPS limit based on the analysis of the trend plot ofFIG. 3 . Then, the process proceeds to step 616. - At
step 616, theserver 213 n determines whether there is any other VM that needs a new IOPS limit. If the answer to thestep 616 is positive, the process proceeds to step 606 so that steps 606-614 are repeated until every VM has a new IOPS limit. Otherwise, the process proceeds to step 618. In embodiments, atstep 618, the IOPS limit(s) have been set and the monitoring may be stopped atstep 620. Alternatively, in embodiments, the IOPS limit(s) may be dynamically monitored on a continuous basis. For example, the subsequent time interval may be set as the current time interval and the process proceeds to step 604 to repeat steps 604-616. - It is noted that, in embodiments, the
server 213 n monitors the throughputs of VMs hosted by theVM host server 208 a and sets new IOPS limits for the VMs at each time interval, i.e., theserver 213 n continuously and dynamically controls the storage IO without further human intervention. In embodiments, the monitored throughput for each VM helps system administrator understand the throughput trend in a preset time interval to thereby assist him to manage the usage of datastore resources in the subsequent time interval. - In embodiments, one or more computing system may be configured to perform one or more of the methods, functions, and/or operations presented herein. Systems that implement at least one or more of the methods, functions, and/or operations described herein may comprise an application or applications operating on at least one computing system. The computing system may comprise one or more computers and one or more databases. The computer system may be a single system, a distributed system, a cloud-based computer system, or a combination thereof.
- It shall be noted that the present invention may be implemented in any instruction-execution/computing device or system capable of processing data, including, without limitation phones, laptop computers, desktop computers, and servers. The present invention may also be implemented into other computing devices and systems. Furthermore, aspects of the present invention may be implemented in a wide variety of ways including software (including firmware), hardware, or combinations thereof. For example, the functions to practice various aspects of the present invention may be performed by components that are implemented in a wide variety of ways including discrete logic components, one or more application specific integrated circuits (ASICs), and/or program-controlled processors. It shall be noted that the manner in which these items are implemented is not critical to the present invention.
- Having described the details of the invention, an
exemplary system 700, which may be used to implement one or more aspects of the present invention, will now be described with reference toFIG. 7 . As illustrated inFIG. 7 ,system 700 includes a central processing unit (CPU) 701 that provides computing resources and controls the computer.CPU 701 may be implemented with a microprocessor or the like, and may also include a graphics processor and/or a floating point coprocessor for mathematical computations.System 700 may also include asystem memory 702, which may be in the form of random-access memory (RAM) and read-only memory (ROM). - A number of controllers and peripheral devices may also be provided, as shown in
FIG. 7 . Aninput controller 703 represents an interface to various input device(s) 704, such as a keyboard, mouse, or stylus. There may also be ascanner controller 705, which communicates with ascanner 706.System 700 may also include astorage controller 707 for interfacing with one ormore storage devices 708 each of which includes a storage medium such as magnetic tape or disk, or an optical medium that might be used to record programs of instructions for operating systems, utilities and applications which may include embodiments of programs that implement various aspects of the present invention. Storage device(s) 708 may also be used to store processed data or data to be processed in accordance with the invention.System 700 may also include adisplay controller 709 for providing an interface to adisplay device 711, which may be a cathode ray tube (CRT), a thin film transistor (TFT) display, or other type of display.System 700 may also include aprinter controller 712 for communicating with aprinter 713. Acommunications controller 714 may interface with one ormore communication devices 715, which enablessystem 700 to connect to remote devices through any of a variety of networks including the Internet, a local area network (LAN), a wide area network (WAN), or through any suitable electromagnetic carrier signals including infrared signals. - In the illustrated system, all major system components may connect to a
bus 716, which may represent more than one physical bus. However, various system components may or may not be in physical proximity to one another. For example, input data and/or output data may be remotely transmitted from one physical location to another. In addition, programs that implement various aspects of this invention may be accessed from a remote location (e.g., a server) over a network. Such data and/or programs may be conveyed through any of a variety of machine-readable medium including, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices that are specially configured to store or to store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices. - Embodiments of the present invention may be encoded upon one or more non-transitory computer-readable media with instructions for one or more processors or processing units to cause steps to be performed. It shall be noted that the one or more non-transitory computer-readable media shall include volatile and non-volatile memory. It shall be noted that alternative implementations are possible, including a hardware implementation or a software/hardware implementation. Hardware-implemented functions may be realized using ASIC(s), programmable arrays, digital signal processing circuitry, or the like. Accordingly, the “means” terms in any claims are intended to cover both software and hardware implementations. Similarly, the term “computer-readable medium or media” as used herein includes software and/or hardware having a program of instructions embodied thereon, or a combination thereof. With these implementation alternatives in mind, it is to be understood that the figures and accompanying description provide the functional information one skilled in the art would require to write program code (i.e., software) and/or to fabricate circuits (i.e., hardware) to perform the processing required.
- It shall be noted that embodiments of the present invention may further relate to computer products with a non-transitory, tangible computer-readable medium that have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind known or available to those having skill in the relevant arts. Examples of tangible computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media; and hardware devices that are specially configured to store or to store and execute program code, such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash memory devices, and ROM and RAM devices. Examples of computer code include machine code, such as produced by a compiler, and files containing higher level code that are executed by a computer using an interpreter. Embodiments of the present invention may be implemented in whole or in part as machine-executable instructions that may be in program modules that are executed by a processing device. Examples of program modules include libraries, programs, routines, objects, components, and data structures. In distributed computing environments, program modules may be physically located in settings that are local, remote, or both.
- One skilled in the art will recognize no computing system or programming language is critical to the practice of the present invention. One skilled in the art will also recognize that a number of the elements described above may be physically and/or functionally separated into sub-modules or combined together.
- It will be appreciated to those skilled in the art that the preceding examples and embodiment are exemplary and not limiting to the scope of the present invention. It is intended that all permutations, enhancements, equivalents, combinations, and improvements thereto that are apparent to those skilled in the art upon a reading of the specification and a study of the drawings are included within the true spirit and scope of the present invention.
Claims (20)
IOPS=(1024/IO block size)*t*(peak value of the throughput),
IOPS=(1024/IO block size)*(peak value of the throughput).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US14/072,919 US9037758B1 (en) | 2013-11-06 | 2013-11-06 | Dynamic virtual machine (VM) input-output per second (IOPS) control based on application IO profiling and VM IO usage trend analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US14/072,919 US9037758B1 (en) | 2013-11-06 | 2013-11-06 | Dynamic virtual machine (VM) input-output per second (IOPS) control based on application IO profiling and VM IO usage trend analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20150127858A1 true US20150127858A1 (en) | 2015-05-07 |
| US9037758B1 US9037758B1 (en) | 2015-05-19 |
Family
ID=53007930
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US14/072,919 Active US9037758B1 (en) | 2013-11-06 | 2013-11-06 | Dynamic virtual machine (VM) input-output per second (IOPS) control based on application IO profiling and VM IO usage trend analysis |
Country Status (1)
| Country | Link |
|---|---|
| US (1) | US9037758B1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150154040A1 (en) * | 2013-12-03 | 2015-06-04 | Fujitsu Limited | Controlling apparatus, and controlling method |
| CN105786591A (en) * | 2016-02-26 | 2016-07-20 | 杭州华三通信技术有限公司 | Method and device for starting virtual machines |
| US20160299693A1 (en) * | 2015-04-08 | 2016-10-13 | Tintri Inc. | Native storage quality of service for virtual machines |
| US20220342705A1 (en) * | 2021-04-27 | 2022-10-27 | EMC IP Holding Company LLC | Greener software defined storage stack |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9448833B1 (en) * | 2015-04-14 | 2016-09-20 | International Business Machines Corporation | Profiling multiple virtual machines in a distributed system |
| US10768827B2 (en) | 2017-04-07 | 2020-09-08 | Microsoft Technology Licensing, Llc | Performance throttling of virtual drives |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7257640B1 (en) * | 2002-04-16 | 2007-08-14 | At&T Corp. | System and method for bandwidth monitoring and allocation in networks |
| US20100274920A1 (en) * | 2005-07-05 | 2010-10-28 | Microsoft Corporation | Adjustment of Transmission Data Rate Based on Data Errors and/or Latency |
| US8688878B1 (en) * | 2012-06-29 | 2014-04-01 | Emc Corporation | Data storage system modeling |
-
2013
- 2013-11-06 US US14/072,919 patent/US9037758B1/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7257640B1 (en) * | 2002-04-16 | 2007-08-14 | At&T Corp. | System and method for bandwidth monitoring and allocation in networks |
| US20100274920A1 (en) * | 2005-07-05 | 2010-10-28 | Microsoft Corporation | Adjustment of Transmission Data Rate Based on Data Errors and/or Latency |
| US8688878B1 (en) * | 2012-06-29 | 2014-04-01 | Emc Corporation | Data storage system modeling |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150154040A1 (en) * | 2013-12-03 | 2015-06-04 | Fujitsu Limited | Controlling apparatus, and controlling method |
| US9430261B2 (en) * | 2013-12-03 | 2016-08-30 | Fujitsu Limited | Controlling apparatus, and controlling method |
| US20160299693A1 (en) * | 2015-04-08 | 2016-10-13 | Tintri Inc. | Native storage quality of service for virtual machines |
| US10248347B2 (en) | 2015-04-08 | 2019-04-02 | Tintri By Ddn, Inc. | Auto allocation of storage system resources to heterogeneous categories of resource consumer |
| US10318197B2 (en) * | 2015-04-08 | 2019-06-11 | Tintri By Ddn, Inc. | Native storage quality of service for virtual machines |
| US20190347023A1 (en) * | 2015-04-08 | 2019-11-14 | Tintri By Ddn, Inc. | Native storage quality of service for virtual machines |
| US10747451B2 (en) | 2015-04-08 | 2020-08-18 | Tintri By Ddn, Inc. | Auto allocation of storage system resources to heterogeneous categories of resource consumer |
| US10949103B2 (en) * | 2015-04-08 | 2021-03-16 | Tintri By Ddn, Inc. | Native storage quality of service for virtual machines |
| CN105786591A (en) * | 2016-02-26 | 2016-07-20 | 杭州华三通信技术有限公司 | Method and device for starting virtual machines |
| US20220342705A1 (en) * | 2021-04-27 | 2022-10-27 | EMC IP Holding Company LLC | Greener software defined storage stack |
| US11928516B2 (en) * | 2021-04-27 | 2024-03-12 | EMC IP Holding Company LLC | Greener software defined storage stack |
Also Published As
| Publication number | Publication date |
|---|---|
| US9037758B1 (en) | 2015-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3577558B1 (en) | Resource management for virtual machines in cloud computing systems | |
| EP3577561B1 (en) | Resource management for virtual machines in cloud computing systems | |
| US9037758B1 (en) | Dynamic virtual machine (VM) input-output per second (IOPS) control based on application IO profiling and VM IO usage trend analysis | |
| US10042663B2 (en) | Maintaining virtual machines for cloud-based operators in a streaming application in a ready state | |
| US9292353B2 (en) | Resource allocation using capacity distribution | |
| US10162684B2 (en) | CPU resource management in computer cluster | |
| US9471386B2 (en) | Allocating resources to tasks in a build process | |
| US9529642B2 (en) | Power budget allocation in a cluster infrastructure | |
| US10606624B2 (en) | Placement of virtual machines on physical hosts | |
| CN109408205B (en) | Task scheduling method and device based on hadoop cluster | |
| US20130007755A1 (en) | Methods, computer systems, and physical computer storage media for managing resources of a storage server | |
| US9372705B2 (en) | Selecting a host for a virtual machine using a hardware multithreading parameter | |
| JP6730522B2 (en) | System and method for allocating input/output bandwidth in storage system | |
| KR102527066B1 (en) | Efficient dynamic resource allocation method and system for maximizing utilization in Kubernetes environment | |
| US10250517B2 (en) | Completion-side client throttling | |
| US9686207B2 (en) | Application service level objective aware demand estimation | |
| US20220237024A1 (en) | Diagonal autoscaling of serverless computing processes for reduced downtime | |
| KR101924467B1 (en) | System and method of resource allocation scheme for cpu and block i/o performance guarantee of virtual machine | |
| US9104481B2 (en) | Resource allocation based on revalidation and invalidation rates | |
| US20200278889A1 (en) | Task management using a virtual node | |
| US20160139940A1 (en) | Systems and methods for creating virtual machine | |
| CN112527482A (en) | Task management method and system based on mobile edge cloud platform | |
| KR20170017183A (en) | Apparatus, method and computer program for controlling virtual cpu | |
| CN106922187A (en) | System, method, and computer program product for low-impact backup | |
| US20160011891A1 (en) | Engine for Virtual Machine Resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: VMWARE, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SARKAR, PRASENJIT;EPPING, DUNCAN GERARDUS CORNELIS ANTONIUS;SINHA, VINEET KUMAR;REEL/FRAME:031552/0279 Effective date: 20131025 |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
| MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
| MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |
|
| AS | Assignment |
Owner name: VMWARE LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:VMWARE, INC.;REEL/FRAME:067102/0395 Effective date: 20231121 |