[go: up one dir, main page]

CN1602480A - Manage memory resources attached to data networks - Google Patents

Manage memory resources attached to data networks Download PDF

Info

Publication number
CN1602480A
CN1602480A CNA028247108A CN02824710A CN1602480A CN 1602480 A CN1602480 A CN 1602480A CN A028247108 A CNA028247108 A CN A028247108A CN 02824710 A CN02824710 A CN 02824710A CN 1602480 A CN1602480 A CN 1602480A
Authority
CN
China
Prior art keywords
memory
virtual
resources
memory resources
physical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CNA028247108A
Other languages
Chinese (zh)
Inventor
亚伯拉罕·希洛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MONOSPHERE Ltd
Original Assignee
MONOSPHERE Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from IL14707301A external-priority patent/IL147073A0/en
Application filed by MONOSPHERE Ltd filed Critical MONOSPHERE Ltd
Publication of CN1602480A publication Critical patent/CN1602480A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Computer And Data Communications (AREA)

Abstract

A computer network includes a plurality of memory nodes (103 a-106 a) each having physical memory resources (121 a). A system management server (150) on a network (100) identifies and collects physical storage (121a) on the network (100) into a virtual storage pool (160). When an application (121) running on a storage client accesses network storage, the system management server (150) allocates a segment of the virtual storage pool (160) to the application. Segments of the virtual memory pool (160) are stored on physical memory (121a) resources on the network (100). The system management server monitors the application's use of network memory and transparently and dynamically reallocates the virtual segments to optimal physical memory resources.

Description

管理附装在数据网络上的存储器资源Manage memory resources attached to data networks

相关申请的交叉参考Cross References to Related Applications

本申请要求根据35 U.S.C.§119的于2001年12月10日提交的第147073号以色列专利申请的优先权。This application claims priority to Israeli Patent Application No. 147073, filed December 10, 2001, under 35 U.S.C. §119.

技术领域technical field

本发明涉及数据网络领域。更具体地说,本发明涉及一种用于动态管理附装在数据网络上的存储器资源并将它们动态分配给也连接在所述数据网络上的多个工作站的方法。The present invention relates to the field of data networks. More specifically, the present invention relates to a method for dynamically managing memory resources attached to a data network and dynamically allocating them to a plurality of workstations also connected to said data network.

背景技术Background technique

在典型的网络计算环境中,可用存储器的数量以许多兆兆字节计算,管理企业级的这种存储器的复杂性也使得对其有效使用变得复杂了。在整个企业中,许多不同版本的相同计算机文件使用户的硬盘混乱不堪。快速检验存储器的使用的努力面临实质性的实现问题。实现一般的存储器分配策略并从企业的角度来进行存储器使用分析也非常复杂。In a typical network computing environment, the amount of memory available is measured in many terabytes, and the complexity of managing this memory at an enterprise level also complicates its efficient use. Across the enterprise, many different versions of the same computer files cluttered users' hard drives. Efforts to quickly verify memory usage face substantial implementation problems. Implementing general memory allocation policies and performing memory usage analysis from an enterprise perspective is also complex.

最近几年,企业遇到了不将其所有存储器资源集中在一起就不能有效实现并管理集中存储策略的问题。此外,出现了不同文件版本之间的不一致性且使有效更新变得难以进行。In recent years, enterprises have encountered the problem of being unable to effectively implement and manage a centralized storage strategy without centralizing all of their storage resources together. Furthermore, inconsistencies between different file versions arise and make efficient updates difficult.

在现有技术中,中央专用文件服务器用作网络的计算机存储器的仓库。如果文件数量大,则可以将该文件服务器分布到多个计算机系统中。然而,随着计算机存储器的容量的增大,将专用文件服务器用于存储显示潜在着瓶颈问题。将许多文件发送到中央专用文件服务器并从它发送许多文件所需的数据吞吐量是网络拥塞的一个主要因素。In the prior art, a central dedicated file server was used as a repository for the network's computer memory. If the number of files is large, the file server can be distributed among several computer systems. However, as the capacity of computer memory increases, the use of dedicated file servers for storage displays a potential bottleneck. The data throughput required to send many files to and from a central dedicated file server is a major contributor to network congestion.

当需求超过一定限度时,将计算机存储器附装到专用文件服务器上的成本以及管理该存储器的复杂性均迅速增大。需要频繁备份该存储器的内容对专用文件服务器产生了更重的负载。The cost of attaching computer storage to a dedicated file server and the complexity of managing that storage grows rapidly when the need exceeds a certain limit. The need to frequently back up the contents of this memory places a heavier load on the dedicated file server.

随着对文件服务器的负载的增大,其操作系统的大部分需专门用于服务器本身的内部管理。由于为了增加可用存储器而添加了更多的硬件部件,所以管理文件服务器存储器的复杂性也增加了。As the load on the file server increases, a large portion of its operating system needs to be dedicated to the internal management of the server itself. The complexity of managing file server storage increases as more hardware components are added to increase available storage.

传统存储器设施不能有效分配存储器资源,因为它们未考虑到访问特定数据项目的频率。例如,在电子邮件应用程序中,访问收件箱文件夹的频率比访问删除项目文件夹的频率高得多。此外,在许多情况中,对服务器静态分配存储器资源产生了其他服务器可以使用的可用存储器不能被充分利用的情况。Conventional memory facilities cannot efficiently allocate memory resources because they do not take into account the frequency with which particular data items are accessed. For example, in an email application, the Inbox folder is accessed much more frequently than the Deleted Items folder. Furthermore, in many cases, static allocation of memory resources to servers creates a situation where the available memory available to other servers is underutilized.

传统存储器分配系统的另一个缺点是服务质量(QoS)低。这意味着,需要大量计算机资源的应用程序缺乏资源,而所需的存储器资源被分配给不强烈需要的应用程序。此外,无效的存储器管理和分配通常导致存储器崩溃,这还导致使用崩溃的存储器的应用程序也崩溃。这也被称为系统停机时间(应用程序由于发生故障而无响应的时间)。当应该维护、升级、添加或移除存储器资源时,产生了传统存储器管理系统的另一个缺点。在这些情况中,几个应用程序(甚或所有应用程序)均应该被挂起,这导致系统停机时间进一步延长。Another disadvantage of conventional memory allocation systems is low quality of service (QoS). This means that applications that require a lot of computer resources are starved of resources, while the required memory resources are allocated to applications that do not require them as strongly. Furthermore, ineffective memory management and allocation often leads to memory corruption, which also causes applications using the corrupted memory to also crash. This is also known as system downtime (time during which an application is unresponsive due to a failure). Another disadvantage of conventional memory management systems arises when memory resources should be maintained, upgraded, added or removed. In these situations, several applications (or even all applications) should be suspended, which leads to further extended system downtime.

因此,需要一种在数据网络上有效管理存储器资源并分布文件的新方法。在当前的技术状态下,对于数据交换,在许多磁盘之间有效分布数据是更好的解决方案。Therefore, there is a need for a new method of efficiently managing memory resources and distributing files over a data network. In the current state of technology, for data exchange, efficiently distributing data across many disks is a better solution.

因此,本发明的一个目的是提供一种用于动态管理并分配存储器资源的方法,该方法克服了现有技术的缺陷。It is therefore an object of the present invention to provide a method for dynamically managing and allocating memory resources which overcomes the drawbacks of the prior art.

本发明的另一个目的是提供一种用于动态管理并分配存储器资源的方法,该方法减少了未使用的存储器资源的数量。Another object of the present invention is to provide a method for dynamically managing and allocating memory resources which reduces the amount of unused memory resources.

本发明的又一个目的是提供一种用于动态管理并分配存储器资源的方法,该方法改善了提供给使用存储器资源的应用程序的服务质量。Yet another object of the present invention is to provide a method for dynamically managing and allocating memory resources which improves the quality of service provided to applications using the memory resources.

本发明的又一个目的是提供一种用于动态管理并分配存储器资源的方法,该方法通过缩短系统停机时间改善了应用程序使用存储器资源的可靠性。Yet another object of the present invention is to provide a method for dynamically managing and allocating memory resources that improves the reliability of memory resources used by applications by reducing system downtime.

本发明的又一个目的是提供一种用于动态管理并分配存储器资源的方法,该方法动态平衡每个应用程序在存储器资源之间施加的负载。Yet another object of the present invention is to provide a method for dynamically managing and allocating memory resources that dynamically balances the load imposed by each application program among the memory resources.

本发明的又一个目的是提供一种根据每个应用程序的实际存储器需要来动态管理并分配存储器资源的方法。Yet another object of the present invention is to provide a method for dynamically managing and allocating memory resources according to the actual memory needs of each application.

发明内容Contents of the invention

本发明涉及一种用于动态管理附装到数据网络上的存储器资源并将该存储器资源动态分配给用户执行的、通过接入点连接到数据网络上的应用程序的方法。周期性地监测分配给每个应用程序的物理存储器资源以及物理存储器资源的性能。一个或者多个物理存储器资源由聚集在虚拟存储器仓库中的相应虚拟存储空间表示。周期性地监测每个应用程序的物理存储器要求。将每个物理存储器资源分割为多个物理存储器分段,每个物理存储器分段的性能属性对应于其物理存储器资源的性能。该仓库被分割为多个虚拟存储器分段,并将每个物理存储器分段映像到具有相同性能属性的相应虚拟存储器分段。对于每个应用程序,引入虚拟存储器资源,该虚拟存储器资源包括根据其相应物理存储器分段的性能属性和要求而为该应用程序优化的各虚拟存储器分段的组合。通过把该组合中的每个虚拟存储器分段重新分配给相应物理存储器分段,对该应用程序重新分配物理存储空间。The invention relates to a method for dynamically managing memory resources attached to a data network and dynamically allocating the memory resources to user-executed applications connected to the data network through an access point. The physical memory resources allocated to each application and the performance of the physical memory resources are periodically monitored. One or more physical storage resources are represented by corresponding virtual storage spaces aggregated in a virtual storage repository. Periodically monitor the physical memory requirements of each application. Each physical memory resource is partitioned into a plurality of physical memory segments, and the performance attribute of each physical memory segment corresponds to the performance of its physical memory resource. The repository is partitioned into multiple virtual memory segments, and each physical memory segment is mapped to a corresponding virtual memory segment with the same performance attributes. For each application, a virtual memory resource is introduced that includes a combination of virtual memory segments optimized for that application according to the performance attributes and requirements of their corresponding physical memory segments. Physical storage space is reallocated for the application by reallocating each virtual memory segment in the combination to a corresponding physical memory segment.

优选地,用于评估性能的参数是应用程序使用存储在物理存储器资源内的数据/数据文件的级别、物理存储器资源的可靠性、物理存储器资源上的可用存储空间、对存储在物理存储器资源上的数据的存取时间以及计算机执行该应用程序与物理存储器资源的接入点之间进行数据交换的延迟。重复评估每个物理存储器资源的性能,并对每个应用程序的物理存储器要求进行监测。将每个虚拟存储器分段重新分配给另一相应物理存储器分段是根据性能和/或要求的变化而动态改变的。Preferably, the parameters used to evaluate performance are the level of data/data files stored in the physical memory resource used by the application, the reliability of the physical memory resource, the available storage space on the physical memory resource, the The access time of the data and the delay of data exchange between the computer executing the application program and the access point of the physical memory resource. Repeatedly evaluate the performance of each physical memory resource and monitor each application's physical memory requirements. The reallocation of each virtual memory segment to another corresponding physical memory segment is dynamically changed according to changes in performance and/or requirements.

通过定义多个存储器节点来进行评估,每个存储器节点表示物理存储器资源与其连接的接入点。对与每个存储器节点有关的一个或者多个参数进行监测,并且为每个存储器节点指定动态分数。The evaluation is performed by defining a number of memory nodes, each representing an access point to which a physical memory resource is connected. One or more parameters associated with each memory node are monitored and a dynamic score is assigned to each memory node.

在一个方面中,对每个存储器节点指定存储优先权。对具有较高存储优先权值的一组存储器节点重新分配与具有执行优先权的应用程序有关的每个虚拟存储器分段。动态监测每个存储器节点的性能,并根据监测结果,改变存储器节点的优先权。每当要求时,就改变对每个虚拟存储器分段的重新分配。In one aspect, storage priorities are assigned to each memory node. Each segment of virtual memory associated with an application having execution priority is reallocated to a set of memory nodes having a higher storage priority value. Dynamically monitor the performance of each storage node, and change the priority of the storage node according to the monitoring results. The reallocation of each virtual memory segment is changed whenever required.

通过将数据文件的复制本存储到几个不同存储器节点上并允许应用程序对存储在具有最佳性能的存储器节点内的复制本进行存取,可以缩短应用程序对所需数据块的存取时间。By storing copies of data files on several different storage nodes and allowing the application to access the copy stored in the storage node with the best performance, the application access time for the required data blocks can be shortened .

通过根据物理存储器资源的添加/移除来更新该仓库的内容、评估每个所添加的物理存储器资源的性能以及根据该性能来动态改变将至少一个虚拟存储器分段重新分配给从所添加的物理存储器资源获得的物理存储器分段和/或另一相应物理存储器分段,以对当前执行的应用程序透明的方式,向数据网络添加/从数据网络移除物理存储器资源。Reallocating at least one virtual memory segment to the added physical memory resource by updating the contents of the repository according to the addition/removal of physical memory resources, evaluating the performance of each added physical memory resource, and dynamically changing according to the performance. The physical memory segment obtained by the memory resource and/or another corresponding physical memory segment, adds/removes the physical memory resource to/from the data network in a manner transparent to the currently executing application.

通过从应用程序发送请求,使得该请求指定请求数据在虚拟存储器资源内的位置,可以对虚拟存储器资源进行数据读操作。请求数据在虚拟存储器资源内的位置被映像到至少含有一部分请求数据的至少一个存储器节点的池。从该池中选择具有执行该请求的最短响应时间的一个或多个存储器节点。将该请求送到具有最低数据交换负载的所选择的存储器节点并且允许该应用程序从所选择的存储器节点读取请求数据。A data read operation may be performed on a virtual memory resource by sending a request from an application program such that the request specifies the location of the requested data within the virtual memory resource. The location of the requested data within the virtual memory resource is mapped to a pool of at least one memory node containing at least a portion of the requested data. One or more memory nodes having the shortest response time to execute the request are selected from the pool. The request is sent to the selected memory node with the lowest data exchange load and the application is allowed to read the requested data from the selected memory node.

通过从应用程序发送请求,使得该请求确定要写入的请求数据以及该数据应当在虚拟存储器资源上写入的位置,可以对虚拟存储器资源进行数据写操作。创建用于存储数据的可能存储器节点的池。从该池中选择至少一个存储器节点,该存储器节点在数据网络上的物理位置具有执行该请求的最短响应时间。将该请求送到具有最低数据交换负载的所选择的存储器节点并且允许该应用程序将数据写入所选择的存储器节点。A data write operation may be performed on a virtual memory resource by sending a request from an application program such that the request determines the requested data to be written and where the data should be written on the virtual memory resource. Create a pool of possible storage nodes for storing data. At least one memory node is selected from the pool whose physical location on the data network has the shortest response time to execute the request. The request is sent to the selected memory node with the lowest data exchange load and the application is allowed to write data to the selected memory node.

利用链接到至少一个存储器节点并作为应用程序与不可存取的存储器资源之间的中介对应用程序不可存取的物理存储器资源进行存取的计算机,每个应用程序均可以对每个存储器节点进行存取。Each application may access each memory node using a computer linked to at least one memory node and providing access to physical memory resources inaccessible to the application as an intermediary between the application and the inaccessible memory resource access.

优选地,对每个应用程序评估每个中介的数据吞吐量性能,并且根据评估结果,将为每个应用程序提供对不可存取的存储器资源的可存取性所需的负载在两个或者更多个中介之间动态分布。Preferably, the data throughput performance of each intermediary is evaluated for each application, and based on the results of the evaluation, each application will be provided with a load required for accessibility to inaccessible memory resources within two or Dynamic distribution among more intermediaries.

通过对两个或者更多个存储器节点重新分配对应于应用程序的虚拟存储器分段,对每个应用程序重新分配物理存储空间,使得根据它们的相应分数在两个或者更多个存储器节点之间动态分布负载,从而在两个或者更多个存储器节点之间达到负载平衡。Reallocate physical storage space for each application by reallocating virtual memory segments corresponding to applications to two or more memory nodes such that the two or more memory nodes are allocated according to their respective scores Dynamically distribute load to achieve load balancing between two or more storage nodes.

通过连续或周期性地监测实际物理存储空间的要求等级、在应用程序实际需要该物理存储空间的时间周期内根据要求等级对应用程序分配实际物理存储空间以及根据要求等级的变化来动态改变分配等级,可以对每个应用程序重新分配物理存储器资源。By continuously or periodically monitoring the requirement level of the actual physical storage space, allocating the actual physical storage space to the application program according to the requirement level during the time period when the application program actually needs the physical storage space, and dynamically changing the allocation level according to the change of the requirement level , physical memory resources can be reallocated per application.

本发明还涉及一种用于动态管理附装到数据网络上的存储器资源并将该存储器资源动态分配给用户执行的、通过接入点连接到数据网络上的应用程序并根据上述方法运行的系统。The invention also relates to a system for dynamically managing memory resources attached to a data network and dynamically allocating the memory resources to applications executed by users connected to the data network through an access point and operating according to the method described above .

附图说明Description of drawings

根据以下参考附图对本发明的优选实施例所做的说明性而非限制性的描述,可以更好地理解本发明的上述以及其他特征和优点,附图包括:These and other features and advantages of the present invention will be better understood from the following illustrative, non-limiting description of preferred embodiments of the invention with reference to the accompanying drawings, which include:

图1示意性地示出根据本发明的优选实施例的用于动态管理存储器资源并对连接到数据网络上的应用服务器/工作站动态分配存储器资源的系统的体系结构;FIG. 1 schematically shows the architecture of a system for dynamically managing memory resources and dynamically allocating memory resources to application servers/workstations connected to a data network according to a preferred embodiment of the present invention;

图2示意性地示出根据本发明的优选实施例的物理存储器资源和虚拟存储器资源的结构以及它们之间的映像;以及Fig. 2 schematically shows the structure of physical storage resources and virtual storage resources and the mapping between them according to a preferred embodiment of the present invention; and

图3A和3B示意性地示出在根据本发明的优选实施例的用于动态管理存储器资源并对连接到数据网络上的应用服务器/工作站动态分配存储器资源的系统内执行的读操作和写操作。3A and 3B schematically illustrate read and write operations performed within a system for dynamically managing memory resources and dynamically allocating memory resources to application servers/workstations connected to a data network according to a preferred embodiment of the present invention .

具体实施方式Detailed ways

本发明包括下面的部件:The present invention comprises following parts:

存储器域监督器,位于系统管理服务器上,用于管理存储器分配策略并对存储器客户机分配存储器;a memory domain supervisor, located on the system management server, for managing memory allocation policies and allocating memory to memory clients;

存储器节点代理,位于其硬盘上具有可用存储空间的每台计算机上;以及a storage node agent, located on each computer that has available storage space on its hard disk; and

存储器客户机,位于需要使用存储空间的每台计算机上。A storage client, located on each computer that needs to use storage space.

以下更将详细说明这些部件中每个部件的任务。The tasks of each of these components are described in more detail below.

图1示意性地示出根据本发明的优选实施例的用于动态管理存储器资源并对连接到数据网络上的应用服务器/工作站动态分配存储器资源的系统的体系结构。数据网络100包括局域网(LAM)101,局域网101包括:网络管理器102;多个工作站103至106,分别具有本地存储器103a至106a;以及多个网区存储器(Network-Area-Storage(NAS))服务器110和111,分别含有大量存储空间,用于LAN用途。NAS服务器110和111与连接至LAN100相关并在其上运行工作站102至105所使用的应用程序的应用服务器121至123进行连续的通信(通过通信通路170)。通信通路170用于临时存储运行LAN101上的工作站运行应用程序所需的数据文件。应用服务器121至123可以含有它们自己(本地存储器)的硬盘121a,或者它们可以通过使用其几个存储磁盘141至143来使用由外部存储区域网络(Storage AreaNetwork(SAN))140所提供的存储服务。将独立存储器资源(诸如硬盘的物理存储器部件)与网络的每个接入点称为存储器节点。Fig. 1 schematically shows the architecture of a system for dynamically managing memory resources and dynamically allocating memory resources to application servers/workstations connected to a data network according to a preferred embodiment of the present invention. The data network 100 includes a local area network (LAM) 101, and the local area network 101 includes: a network manager 102; a plurality of workstations 103 to 106, respectively having local storage 103a to 106a; and a plurality of network area storage (Network-Area-Storage (NAS)) Servers 110 and 111, each containing a large amount of storage space, are used for LAN purposes. NAS servers 110 and 111 are in continuous communication (via communication path 170 ) with application servers 121 to 123 connected to LAN 100 and running applications used by workstations 102 to 105 thereon. Communication path 170 is used to temporarily store data files required to run applications on workstations on LAN 101 . Application servers 121 to 123 may contain their own (local storage) hard disk 121a, or they may use storage services provided by an external Storage Area Network (Storage Area Network (SAN)) 140 by using their several storage disks 141 to 143 . Each access point of an independent memory resource (a physical memory component such as a hard disk) to the network is called a memory node.

根据现有技术,应用服务器121至123分别将其应用程序的数据存储到其自己的相应硬盘121a上(如果足够大),或者存储到由SAN140分配的相应磁盘141至143上。为了克服未用存储空间、系统停机时间以及服务质量不够高的缺陷,对网络管理器101添加管理服务器150。管理服务器150识别连接至网络100相关的所有物理存储器资源(即所有硬盘),并将它们集合为一个虚拟存储器池160,实际上,虚拟存储器池160是由多个利用动态处理并评估的预定判据的分布在物理存储器资源之中的分段实现的,使得该分布对每个应用程序是透明的。此外,管理服务器150监测(通过运行安装在其内的存储器域监督器)网络上的工作站103至106当前使用的所有各种应用程序。因此,服务器150可以检测每个应用程序实现占用运行该应用程序的应用服务器多少磁盘空间。利用该知识和判据,根据每个应用程序的实际需要和使用等级,服务器150对每个应用程序重新分配虚拟存储器资源。为了对网络管理器102产生动态指示,服务器150处理采集的知识,以在运行的应用程序之间调整和重新分配可用存储空间,同时对每个应用程序提供该应用程序正确运行所希望的虚拟存储空间量。设置服务器150,以便它与LAN101与应用服务器121至123之间的网络通信通路171并行。这种配置确保服务器150不是流过通信通路171的数据的瓶颈,因此,消除了数据拥塞。According to the prior art, the application servers 121 to 123 respectively store the data of their application programs on their own corresponding hard disk 121 a (if large enough), or on corresponding disks 141 to 143 allocated by the SAN 140 . In order to overcome the disadvantages of unused storage space, system downtime and insufficient quality of service, a management server 150 is added to the network manager 101 . The management server 150 identifies all physical memory resources (i.e., all hard disks) connected to the network 100 and aggregates them into a virtual memory pool 160. The distribution of data is implemented in segments among physical memory resources, making this distribution transparent to each application. Furthermore, the management server 150 monitors (by running the storage domain supervisor installed therein) all of the various applications currently used by the workstations 103 to 106 on the network. Thus, server 150 can detect how much disk space each application implementation occupies on the application server running the application. Using this knowledge and criteria, server 150 reallocates virtual memory resources to each application based on each application's actual needs and usage levels. In order to generate dynamic instructions to the network manager 102, the server 150 processes the gathered knowledge to adjust and reallocate the available storage space among running applications, while providing each application with the virtual storage desired by that application to function properly amount of space. The server 150 is arranged so that it runs parallel to the network communication path 171 between the LAN 101 and the application servers 121 to 123 . This configuration ensures that server 150 is not a bottleneck for data flowing through communication path 171, thus eliminating data congestion.

重新分配过程基于许多应用程序尽管占用大量磁盘资源,但是实际上仅使用部分资源的事实。应用程序未使用的剩余资源仅是应用程序知道其存在时所需的,而不是在其上面运行。例如,应用程序可以占用15GB的内存,而用于安装和数据文件的磁盘实际使用的只有10GB。为了正确运行,该应用程序要求剩余的5GB在其分配磁盘上可用,但是几乎从未(或永远不)使用它们。重新分配过程接管磁盘资源的这样未用部分,并将它们分配给实际运行需要它们的应用程序。这样,网络的虚拟存储卷的大小可以大于实际物理存储空间。这样就将网络的灵活性提高到其操作系统的物理存储空间的格式化能力的极限。根据要求(动态地)并且仅在该应用程序实际需要的时间周期内对每个应用程序分配实际物理存储空间。连续或者周期性地监测要求等级,而且如果检测到要求等级降低,则相应地对该应用程序减少分配的物理存储空间的数量,且可以将它分配给当前提高其要求等级的其他应用程序。对于对每个应用程序分配虚拟存储器资源同样可以这样做。The reallocation process is based on the fact that many applications, despite being disk intensive, actually only use some of the resources. The remaining resources not used by the application are only needed by the application when it is aware of its existence, not running on it. For example, an application can take up 15GB of memory, while only 10GB is actually used by the disk for installation and data files. To run correctly, the application requires the remaining 5GB to be available on its allocated disk, but they are almost never (or never) used. The reallocation process takes over such unused portions of disk resources and allocates them to the applications that actually need them to run. In this way, the size of the virtual storage volume of the network can be larger than the actual physical storage space. This increases the flexibility of the network to the limit of the formatting capabilities of its operating system's physical storage space. Each application is allocated actual physical storage space on demand (dynamically) and only for the period of time that the application actually needs. The demand level is monitored continuously or periodically, and if a decrease in the demand level is detected, the amount of allocated physical storage space is correspondingly reduced for the application and can be allocated to other applications currently increasing their demand level. The same can be done for allocating virtual memory resources to each application.

该系统可以实现的另一可选特征是其流动性(liquidity)—它是系统应该分配多少附加存储器资源供应用程序立即使用的指示。流动性提供更好的存储器分配性能,而且可以确保应用程序不会因为存储器要求的未预期增加而用尽存储器资源。在应用程序用尽可用存储器资源之前,存储卷使用指示器向系统管理器告警。Another optional feature that the system can implement is its liquidity—it is an indication of how much additional memory resources the system should allocate for immediate use by applications. Fluidity provides better memory allocation performance and ensures that applications do not run out of memory resources due to unexpected increases in memory requirements. The storage volume usage indicator alerts the system manager before applications exhaust available memory resources.

该系统的又一可选特征是其可存取性—即使这些存储设备中的一些存储设备只能被网络内的有限数量的计算机存取,但是它仍可以使应用服务器对所有网络存储设备(存储器节点)进行存储。这是通过把可以存取不可存取的磁盘的计算机用作中介并使它们对请求不可存取数据的应用程序进行存取的方式来实现的。对于每个应用程序,具体评估每个中介的数据吞吐量性能(即,该中介在给定时间周期内成功处理的数据量),而且根据评估结果,在不同中介之间对每个应用程序动态分配实现可存取性所需的负载。Yet another optional feature of the system is its accessibility—even though some of these storage devices can only be accessed by a limited number of computers in the network, it still enables the application server to have access to all network storage devices ( storage nodes) for storage. This is accomplished by using computers that can access the inaccessible disks as intermediaries and giving them access to applications that request the inaccessible data. For each application, the data throughput performance of each intermediary (i.e., the amount of data successfully processed by the intermediary in a given period of time) is evaluated in detail, and based on the evaluation results, the performance of each intermediary is dynamically evaluated among different intermediaries. Distribute the load needed to achieve accessibility.

为了确保其资源被免除的应用程序仍可以无故障地运行,服务器150分别对应用服务器121、122和123创建虚拟存储卷161、162和163(在虚拟存储器池160中)。这些虚拟存储卷反映为虚拟磁盘121b、122b和123b。这意味着,即使应用程序未拥有运行所需的全部物理磁盘资源,它仍从网络管理器102接收到其可以使用所有这些资源的指示,而实际上,其未用资源被分配给其他应用程序。因此,应用服务器只知道其虚拟磁盘而非其物理磁盘的大小。由于每个应用程序的资源要求经常变化,所以应用服务器所看到的虚拟磁盘的大小也在变化。将每个虚拟存储卷分割为预定存储分段(“块”),通过在相应物理存储器资源之间分配预定存储分段,可以将该预定存储分段动态映像回到物理存储器资源(例如,磁盘121a、141至143)。To ensure that applications whose resources are exempted can still run without failure, server 150 creates virtual storage volumes 161, 162 and 163 (in virtual storage pool 160) for application servers 121, 122 and 123, respectively. These virtual storage volumes are reflected as virtual disks 121b, 122b and 123b. This means that even though an application does not own all the physical disk resources it needs to run, it still receives an indication from the network manager 102 that it can use all of these resources, when in fact its unused resources are allocated to other applications . Therefore, the application server only knows the size of its virtual disk and not its physical disk. Because the resource requirements of each application change frequently, the size of the virtual disk as seen by the application server also changes. Each virtual storage volume is partitioned into predetermined storage segments ("chunks") that can be dynamically mapped back to physical memory resources (e.g., disks) by allocating the predetermined storage segments between corresponding physical memory resources. 121a, 141 to 143).

对每个存储器节点设置存储器节点代理,存储器节点代理是软件部件,它执行对所分配的物理存储器资源与虚拟存储器资源之间的数据交换的重新分配。根据本发明的优选实施例,链接到最终用户的工作站的每个存储器节点的资源也被添加到虚拟存储器池160上。通过定义分别连接到相应物理存储器资源上的多个存储器节点130a至130i来进行映像。评估每个存储器节点,而且每个存储器节点以从预定判据获得的性能参数(例如,该节点上的可用物理存储器、通过数据网络到达该节点的数据延迟、对连接到该存储器节点上的磁盘的存取时间等)为特征。A storage node agent is provided for each storage node, and the storage node agent is a software component that performs reallocation of data exchange between allocated physical and virtual memory resources. According to a preferred embodiment of the present invention, the resources of each memory node linked to the end user's workstation are also added to the virtual memory pool 160 . Mapping is performed by defining a plurality of memory nodes 130a to 130i respectively connected to corresponding physical memory resources. Each memory node is evaluated with performance parameters obtained from predetermined criteria (e.g., physical memory available on the node, latency of data reaching the node over a data network, response to disks connected to the memory node) access time, etc.) as a feature.

为了优化重新分配过程,服务器150动态评估每个存储器节点,并且以对该应用程序透明的方式,对于每个应用程序,在发现最适合该应用程序的存储器节点之间分布(通过分配)对应于该应用程序的物理存储器分段。将应用程序发出的存取其数据文件的每个请求发送到当前含有这些数据文件的相应存储器节点。重复该评估过程,并根据评估结果将数据文件从一个节点移到另一个节点。To optimize the reallocation process, the server 150 dynamically evaluates each memory node, and in a manner transparent to the application, for each application, distributes (by allocating) among the memory nodes found to be most suitable for the application corresponding to The application's physical memory segment. Each request made by an application to access its data files is sent to the corresponding storage node that currently contains those data files. The evaluation process is repeated, and data files are moved from one node to another based on the evaluation results.

管理控制台164控制服务器150的运行,管理控制台150通过LAN/WAN165与服务器150通信并将动态指示提供给网络管理器102。Management console 164 controls the operation of server 150 , and management console 150 communicates with server 150 via LAN/WAN 165 and provides dynamic instructions to network manager 102 .

服务器150包括指向与系统内的每个文件对应的、虚拟存储器池160内的位置的指针,因此,发出文件请求的应用程序不需要知道其实际物理位置。虚拟存储器池160维护一组将虚拟存储空间映像到位于整个网络上的不同磁盘(存储器节点)上的一组存储器物理卷的表。Server 150 includes pointers to locations within virtual memory pool 160 corresponding to each file within the system, so applications making file requests need not know their actual physical location. Virtual storage pool 160 maintains a set of tables that map virtual storage space to a set of storage physical volumes located on different disks (storage nodes) throughout the network.

利用虚拟存储器池160,任何客户机应用程序均可以存取连接至网络相关的每个存储磁盘上的每个文件。在转发数据请求期间,客户机应用程序识别自身,因此可以从虚拟存储器池160中的正确表中提取其存取的安全等级。Using the virtual storage pool 160, any client application can access every file on every storage disk associated with the network. During forwarding of a data request, the client application identifies itself so it can extract the security level of its access from the correct table in the virtual memory pool 160 .

图2示意性地示出根据本发明的优选实施例的物理存储器资源和虚拟存储器资源的结构以及它们之间的映像。与每个应用程序有关的每个虚拟存储卷(例如161)被分割为相等的存储“块”,该存储块又被细分为分段,使得每个分段与最佳存储器节点有关(通过连续的评估)。通过其相应最佳存储器节点,块的每个分段被映像到位于与该节点相关的磁盘的相应部分上的“小块”。从该图可以看出,每个块可以被映像到每个分别具有不同性能且位于数据网络上的不同位置的多个磁盘(分布在多个磁盘之间)上。Fig. 2 schematically shows the structures of physical memory resources and virtual memory resources and the mapping between them according to a preferred embodiment of the present invention. Each virtual storage volume (e.g., 161) associated with each application is partitioned into equal storage "blocks," which in turn are subdivided into segments such that each segment is associated with the optimal storage node (via continuous evaluation). Each segment of a block is mapped by its corresponding optimal memory node to a "chunk" located on the corresponding portion of the disk associated with that node. It can be seen from this figure that each block can be mapped to multiple disks (distributed among multiple disks) each having different performance and located at different locations on the data network.

本发明所建议的分层体系结构使存储网络具有可扩展性,同时在基本保持其性能。网络被分割为互相相关连接的区域(例如分开的LAN)。每个预定区域内所选择的计算机维护其把虚拟存储空间映像到位于该区域内的一组物理存储器资源的本地路由表。每当需要对未映像的存储卷进行存取时,计算机在虚拟存储器池160内搜索请求的存储卷的位置,然后,存取其数据。每次在存储区域内的数据发生变化时,更新该本地路由表。对于所有区域,只有虚拟存储器池160保持元数据(即,与所存储的数据文件的属性、结构以及位置有关的数据)变化的整个情况。这样,特别是对于大存储网络,可以将为了对网络上的任何存储器节点上的文件进行存取而应该对虚拟存储器池160进行存取的次数降到最低,以及可以将更新本地路由表所需的元数据业务量降到最低。The layered architecture suggested by the present invention makes the storage network expandable while maintaining its performance substantially. The network is divided into areas (eg separate LANs) that are connected to each other. Selected computers within each predetermined region maintain their local routing tables that map virtual storage space to a set of physical memory resources located within that region. Whenever an unmapped storage volume needs to be accessed, the computer searches for the location of the requested storage volume within the virtual storage pool 160, and then accesses its data. The local routing table is updated each time the data in the storage area changes. Of all regions, only the virtual memory pool 160 maintains a complete picture of changes in metadata (ie, data related to the attributes, structure, and location of stored data files). In this way, especially for large storage networks, the number of times that virtual storage pool 160 should be accessed in order to access files on any storage node on the network can be minimized, and the number of times required to update the local routing table can be minimized. The amount of metadata traffic is reduced to a minimum.

可以利用独立磁盘冗余阵列(RAID—将同一个数据冗余存储到多个硬盘上(即不同位置))来实现物理存储器资源。保存文件的多个副本是一种更加节省成本的方法,因为不存在再存入它们所产生的操作延迟,而且可以立即使用这些文件的备份。Physical memory resources can be implemented using Redundant Array of Independent Disks (RAID—redundantly storing the same data on multiple hard disks (ie, different locations)). Saving multiple copies of files is a more cost-effective method because there is no operational delay in redistributing them, and backups of these files are available immediately.

图3A和3B示意性地示出在根据本发明的优选实施例的用于动态管理存储器资源并对连接到数据网络上的应用服务器/工作站动态分配存储器资源的系统内执行的读操作和写操作。3A and 3B schematically illustrate read and write operations performed within a system for dynamically managing memory resources and dynamically allocating memory resources to application servers/workstations connected to a data network according to a preferred embodiment of the present invention .

在写操作中,用户应用程序(运行在存储器客户机上)发出读某个数据的请求,并对该请求附加3个参数—要读取的虚拟卷、该卷内的请求数据的偏移以及数据长度。通过文件系统转发该请求并对存储器客户机的低等级设备部件进行存取,低等级设备部件通常是磁盘。然后,低等级设备调用块分配器。块分配器利用卷映像表将请求数据(卷规定的以及该请求的偏移参数)的虚拟位置(虚拟存储器池160内的分配虚拟驱动器)变换为网络内实际存储该数据的物理位置(存储器节点)。In a write operation, the user application (running on the storage client) issues a request to read some data, and attaches three parameters to the request—the virtual volume to be read, the offset of the requested data within the volume, and the data length. The request is forwarded through the file system and accessed to a low-level device component of the storage client, usually a disk. Then, the low-level device calls the block allocator. The block allocator uses the volume mapping table to translate the virtual location (the allocation virtual drive within the virtual memory pool 160) of the requested data (volume-specified and the requested offset parameters) into the physical location within the network where the data is actually stored (the storage node ).

通常,存在请求数据被写入网络内的一个以上的位置内的情况。为了判定最好从哪个存储器节点检索数据,存储器客户机周期性地将对所读取的文件的请求发送到网络上的每个存储器节点,然后,测量响应时间。此后,它创建对于存储器客户机的位置具有最短读存取时间(最高优先权)的最佳存储器节点的表。负载均衡器利用该表计算从其检索请求数据的最佳存储器节点。可以从具有最高优先权的存储器节点检索数据。作为一种选择,如果由于存在其他应用程序的并行请求而使具有最高优先权的存储器节点拥塞,则从具有类似或次最佳优先权的的另一个存储器节点检索数据。由于对于每个应用程序连续(或者周期性地)评估每个存储器节点的性能,所以根据评估结果,可以在含有每个应用程序的部分所需数据的所有不同存储器节点之间动态分布数据检索(存储器节点之间的负载均衡)。对于每个应用程序,可以根据评估结果的变化,改变用于每个读操作的存储器节点的组合。Often, there are situations where request data is written to more than one location within the network. To decide which storage node is best to retrieve data from, the storage client periodically sends requests for the read file to each storage node on the network, and then measures the response time. Thereafter, it creates a table of the best memory node with the shortest read access time (highest priority) for the memory client's location. The load balancer uses this table to calculate the best storage node from which to retrieve the requested data. Data can be retrieved from the memory node with the highest priority. Alternatively, if the memory node with the highest priority is congested due to concurrent requests from other applications, data is retrieved from another memory node with similar or next-best priority. Since the performance of each memory node is evaluated continuously (or periodically) for each application, data retrieval can be dynamically distributed among all the different memory nodes that contain part of the data required by each application ( load balancing between storage nodes). For each application, the combination of memory nodes used for each read operation can be changed according to the change of the evaluation result.

确定了检索位置后,在系统内负责I/O操作的RAID控制器通过各种网络通信卡发送请求。然后,它对正确存储器节点进行存取,然后,检索请求数据。After determining the retrieval location, the RAID controller in charge of I/O operation in the system sends requests through various network communication cards. It then accesses the correct memory node and then retrieves the requested data.

利用类似的方法,执行写操作。对从用户应用程序接收的写数据的请求也具有3个参数,但是此时,不是数据长度(在读操作中出现),而是要写入的实际数据。初始步骤相同,直到块分配器从卷映像表提取应该将数据写入其内的精确位置。接着,块分配器利用节点速度结果和使用信息表来检验整个网络上的所有可用存储器节点,并且形成可能存储空间池,用于写入数据。块分配器对创建每个请求的数据块的至少两个复制本分配所需的存储器,以便用户建立新数据文件。Using a similar approach, write operations are performed. The request to write data received from the user application also has 3 parameters, but this time, instead of the data length (which occurs in a read operation), it is the actual data to be written. The initial steps are the same until the block allocator extracts from the volume image table the precise location within which the data should be written. The block allocator then uses the node speed results and the usage information table to examine all available memory nodes across the network and form a pool of possible storage space for writing data. The block allocator allocates the memory required to create at least two copies of each requested data block for the user to create a new data file.

为了从池中选择存储器节点,为了以最有效方式来分配存储器,根据由下面参数所确定的优先权,负载均衡器评估每个远程存储器节点:To select storage nodes from the pool, the load balancer evaluates each remote storage node in order to allocate storage in the most efficient manner, according to the priority determined by the following parameters:

存储器节点上剩余的存储量。Amount of storage remaining on the storage node.

其他应用程序发送到该存储器节点的用于存取数据的其他请求。Other requests sent by other applications to this storage node to access data.

到达该节点的通路上的数据拥塞。Data congestion on the path to the node.

将数据写入具有最高优先权的存储器节点,或者通过连续(或者周期性地)对每个应用程序评估每个存储器节点的性能,将数据写入。对于每个应用程序,根据评估结果(存储器节点之间的负载均衡),在不同(甚或所有)存储器节点之间动态分布数据写操作。对于每个应用程序,可以根据评估结果的变化,改变用于每个写操作的存储器节点的组合。Data is written to the memory node with the highest priority, or by continuously (or periodically) evaluating the performance of each memory node for each application. For each application, data write operations are dynamically distributed among different (or even all) memory nodes according to the evaluation results (load balancing among memory nodes). For each application, the combination of memory nodes used for each write operation can be changed according to the change of the evaluation result.

选择了要使用的存储器节点后,RAID控制器将写请求发送到正确的NAS和SAN设备,并通过各种网络通信卡把数据送给它们。然后,接收数据,并将该数据存储到正确NAS和SAN设备内的正确存储器节点上。After selecting the storage nodes to be used, the RAID controller sends write requests to the correct NAS and SAN devices and sends data to them through various network communication cards. The data is then received and stored on the correct storage nodes within the correct NAS and SAN devices.

由于其用户对存储在网络上的数据的请求持续变化,所以根据改变的存储器请求,动态修改该数据的存储器分布。最后,根据用户对它的要求,优化该数据的实例的数量,并改变其在网络上的不同存储器节点之间的物理位置。这样,该系统本身不断进行调节,直到达到最佳配置。Since its users' requests for data stored on the network are constantly changing, the memory distribution of this data is dynamically modified according to changing memory requests. Finally, the number of instances of this data is optimized and its physical location between different memory nodes on the network is changed according to the user's requirements for it. In this way, the system keeps adjusting itself until an optimal configuration is reached.

根据本发明的优选实施例,将每个文件的多个复制本至少存储到网络上的两个不同节点上,作为以防系统发生故障的备份。对于每个请求的文件,评估存储在与该文件有关的配置表内的文件使用方式。通过评估文件的复制本并将文件的复制本存储到网络上的分开存储器节点上,通过根据评估结果消除存取争用,增大了网络上的数据吞吐量。According to a preferred embodiment of the present invention, multiple copies of each file are stored on at least two different nodes on the network as a backup in case of system failure. For each requested file, the file usage pattern stored in the configuration table associated with that file is evaluated. By evaluating the replica of the file and storing the replica of the file on a separate storage node on the network, data throughput on the network is increased by eliminating access contention based on the evaluation results.

不是利用中央服务器,而是通过在网络的不同节点上同时产生多个文件复制本来进行文件分布。因此,分散分布,而且消除了瓶颈状态。Instead of utilizing a central server, file distribution is performed by simultaneously generating multiple copies of the file on different nodes of the network. Thus, the distribution is decentralized and the bottleneck state is eliminated.

动态地执行映像处理,而不中断应用程序。因此,通过将新存储磁盘登记在虚拟存储器池中,可以将它们添加到数据网络中。Perform image processing dynamically without interrupting applications. Thus, new storage disks can be added to the data network by registering them in the virtual storage pool.

在虚拟存储器池160的表中动态维护关于每个文件的每个复制本的存储位置以及关于包括这些文件的存储器的每个块(硬盘上的小尺寸存储器分段)的更新元数据。The storage location of each replica of each file and updated metadata about each block (small size memory segment on the hard disk) of memory comprising these files is dynamically maintained in tables of the virtual memory pool 160 .

还动态设置不同文件的冗余度等级,其中,在整个网络上的更多个位置复制具有重要数据的文件,这样可以更好地防止存储器发生故障。Redundancy levels for different files are also set dynamically, where files with important data are replicated in more locations across the network, which provides better protection against storage failures.

当然,提供上述例子和描述仅为了说明本发明,而无意在任何方面限制本发明。本技术领域的普通技术人员应当明白,可以以大量方式,采用以上描述的技术中的一种以上的技术实现本发明,它们均属于本发明的范围。Of course, the above examples and descriptions are provided only to illustrate the present invention and are not intended to limit the present invention in any respect. Those skilled in the art should understand that the present invention can be implemented in a large number of ways using more than one of the technologies described above, and they all belong to the scope of the present invention.

Claims (30)

1.一种用于管理网络上的存储器资源的系统,该系统包括:1. A system for managing memory resources on a network, the system comprising: 多个存储器节点,其位于网络上,每个节点分别与物理存储器资源相关;a plurality of memory nodes located on the network, each node being respectively associated with a physical memory resource; 管理服务器,其位于网络上,用于将与该存储器节点相关的物理存储器资源集中为虚拟存储器资源池;以及a management server, located on the network, for concentrating physical memory resources associated with the memory node into a pool of virtual memory resources; and 存储器客户机,用于对由管理服务器集中的池中的虚拟存储器资源进行存取。A storage client for accessing virtual storage resources in a pool centralized by the management server. 2.根据权利要求1所述的系统,其中,该虚拟存储器池包括多个虚拟分段,并且其中,该虚拟分段适于存储在物理存储器资源上。2. The system of claim 1, wherein the virtual memory pool includes a plurality of virtual segments, and wherein the virtual segments are adapted to be stored on physical memory resources. 3.根据权利要求2所述的系统,其中,在虚拟存储卷上设置该虚拟分段,并且其中,对于存储器客户机,该虚拟存储卷表现为物理存储器资源。3. The system of claim 2, wherein the virtual segment is set on a virtual storage volume, and wherein, to a storage client, the virtual storage volume appears as a physical storage resource. 4.根据权利要求1所述的系统,其中,池内的虚拟存储器总数超过网络上的物理存储器资源总数。4. The system of claim 1, wherein the total number of virtual memory within the pool exceeds the total number of physical memory resources on the network. 5.根据权利要求1所述的系统,其中,该管理服务器适于对存储器客户机对虚拟存储器资源的存取进行监测,并且根据该存取,将虚拟存储器资源动态分配给物理存储器资源。5. The system of claim 1, wherein the management server is adapted to monitor access of virtual memory resources by memory clients and dynamically allocate virtual memory resources to physical memory resources based on the access. 6.根据权利要求5所述的系统,其中,物理存储器资源的特征由性能参数表示,并且其中,根据该性能参数以及存储器客户机进行存取的特性,管理服务器将虚拟存储器资源动态分配给物理存储器资源。6. The system of claim 5, wherein physical memory resources are characterized by performance parameters, and wherein the management server dynamically allocates virtual memory resources to physical memory resources. 7.根据权利要求5所述的系统,其中,对于存储器客户机,动态分配是透明的。7. The system of claim 5, wherein dynamic allocation is transparent to memory clients. 8.根据权利要求5所述的系统,其中,该管理服务器适于根据存储器客户机使用虚拟存储器的等级,将虚拟存储器资源动态分配给物理存储器资源。8. The system of claim 5, wherein the management server is adapted to dynamically allocate virtual memory resources to physical memory resources according to the level of use of virtual memory by memory clients. 9.根据权利要求5所述的系统,其中,存储器客户机适于执行多个应用程序,并且其中,管理服务器适于对多个应用程序中的各应用程序对虚拟存储器资源的存取进行监测,并且根据应用程序进行的存取,将虚拟存储器动态分配给多个应用程序中的每个应用程序。9. The system of claim 5, wherein the memory client is adapted to execute a plurality of applications, and wherein the management server is adapted to monitor access to virtual memory resources by each of the plurality of applications , and the virtual memory is dynamically allocated to each of the plurality of application programs according to accesses made by the application programs. 10.根据权利要求1所述的系统,其中,该存储器客户机存取多个虚拟存储器资源所保持的数据,并且其中,存储器客户机还适于测试保持该数据的多个虚拟存储器资源,并识别一组从其存取该数据的最佳虚拟存储器资源。10. The system of claim 1, wherein the memory client accesses data held by a plurality of virtual memory resources, and wherein the memory client is further adapted to test the plurality of virtual memory resources holding the data, and A set of optimal virtual memory resources from which to access the data is identified. 11.根据权利要求10所述的系统,其中,该存储器客户机进一步包括:11. The system of claim 10, wherein the memory client further comprises: 负载均衡器,其适于在从其存取该数据的组中选择虚拟存储器资源。A load balancer adapted to select a virtual memory resource among the groups from which the data is accessed. 12.根据权利要求1所述的系统,其中,网络上的存储器节点对于存储器客户机不可存取,但是对于中介计算机系统可存取,并且其中,管理服务器适于利用中介计算机系统来使存储器客户机对与该存储器节点相关的物理存储器进行存取。12. The system of claim 1 , wherein the memory nodes on the network are inaccessible to the memory client, but are accessible to the intermediary computer system, and wherein the management server is adapted to use the intermediary computer system to make the memory client The machine accesses the physical memory associated with the memory node. 13.根据权利要求1所述的系统,其中,该网络包括多个区域,每个区域包括多个存储器节点,该系统进一步包括:13. The system of claim 1, wherein the network comprises a plurality of regions, each region comprising a plurality of memory nodes, the system further comprising: 计算机系统,具有本地路由表,该本地路由表将虚拟存储器资源池映像到与各区域之一的多个存储器节点相关的物理存储器资源。A computer system has a local routing table that maps a pool of virtual memory resources to physical memory resources associated with a plurality of memory nodes in one of the regions. 14.一种计算机程序产品,该计算机程序产品包括:14. A computer program product, the computer program product comprising: 计算机可读介质,其具有嵌入其内的计算机程序逻辑,用于维护网络上的存储器资源,该网络包括多个存储器节点,每个节点分别与物理存储器资源相关,该网络进一步包括用于存取存储器资源的存储器客户机,该计算机程序逻辑包括:A computer-readable medium having computer program logic embedded therein for maintaining memory resources on a network comprising a plurality of memory nodes each associated with a physical memory resource, the network further comprising means for accessing A memory client of a memory resource, the computer program logic comprising: 管理服务器逻辑,用于将与该存储器节点相关的物理存储器资源集中为虚拟存储器资源池,并用于对存储器客户机提供该池中的虚拟存储器资源。The management server logic is used to concentrate the physical storage resources related to the storage node into a virtual storage resource pool, and to provide the storage client with the virtual storage resources in the pool. 15.根据权利要求14所述的计算机程序产品,其中,虚拟存储器池包括多个虚拟分段,并且其中,该虚拟分段适于存储在物理存储器资源上。15. The computer program product of claim 14, wherein the virtual memory pool includes a plurality of virtual segments, and wherein the virtual segments are adapted to be stored on physical memory resources. 16.根据权利要求15所述的计算机程序产品,其中,在虚拟存储卷上设置虚拟分段,并且其中,对于存储器客户机,该虚拟存储卷表现为物理存储器资源。16. The computer program product of claim 15, wherein virtual segmentation is set on a virtual storage volume, and wherein the virtual storage volume appears to a storage client as a physical storage resource. 17.根据权利要求14所述的计算机程序产品,其中,管理服务器逻辑进一步适于对存储器客户机对存储器资源的存取进行监测,并且根据存取,将虚拟存储器资源动态分配给物理存储器资源。17. The computer program product of claim 14, wherein the management server logic is further adapted to monitor memory client accesses to memory resources and dynamically allocate virtual memory resources to physical memory resources based on the accesses. 18.根据权利要求17所述的计算机程序产品,其中,物理存储器资源的特征由性能参数表示,并且其中,根据该性能参数以及存储器客户机进行存取的特性,管理服务器逻辑将虚拟存储器资源动态分配给物理存储器资源。18. The computer program product of claim 17, wherein physical memory resources are characterized by performance parameters, and wherein the management server logic dynamically assigns virtual memory resources to allocated to physical memory resources. 19.根据权利要求17所述的计算机程序产品,其中,存储器客户机适于执行多个应用程序,并且其中,管理服务器逻辑适于对多个应用程序中的各应用程序对存储器资源的存取进行监测,并且根据应用程序进行的存取,将虚拟存储器资源动态分配给多个应用程序中的每个应用程序。19. The computer program product of claim 17, wherein the memory client is adapted to execute a plurality of applications, and wherein the management server logic is adapted to access memory resources by each of the plurality of applications Monitoring is performed and virtual memory resources are dynamically allocated to each of the plurality of applications based on accesses made by the applications. 20.根据权利要求14所述的计算机程序产品,其中,存储器客户机存取多个虚拟存储器资源所保持的数据,该计算机程序产品进一步包括:20. The computer program product of claim 14, wherein the memory client accesses data held by a plurality of virtual memory resources, the computer program product further comprising: 测试逻辑,用于测试保持该数据的多个虚拟存储器资源,并识别存储器客户机应该从其存取该数据的一组最佳虚拟存储器资源。Test logic for testing a plurality of virtual memory resources holding the data and identifying an optimal set of virtual memory resources from which the memory client should access the data. 21.根据权利要求20所述的计算机程序产品,该计算机程序产品进一步包括:21. The computer program product of claim 20, further comprising: 负载均衡器逻辑,适于在存储器客户机从其存取该数据的组中选择虚拟存储器资源。Load balancer logic adapted to select the virtual memory resource among the group from which the memory client accesses the data. 22.一种用于管理网络上的存储器资源的方法,该方法包括:22. A method for managing memory resources on a network, the method comprising: 识别网络上的多个存储器节点,每个节点分别与物理存储器资源相关;identifying a plurality of memory nodes on the network, each node being individually associated with a physical memory resource; 将与该存储器节点相关的物理存储器资源集中为虚拟存储器资源池;以及concentrating physical memory resources associated with the memory node into a pool of virtual memory resources; and 根据存储器客户机对网络上的存储器资源进行存取,对存储器客户机提供该池中的虚拟存储器资源。Virtual memory resources in the pool are provided to memory clients based on memory clients accessing memory resources on the network. 23.根据权利要求22所述的方法,其中,该虚拟存储器池包括多个虚拟分段,并且其中,在物理存储器资源之间分布该虚拟分段。23. The method of claim 22, wherein the virtual memory pool includes a plurality of virtual segments, and wherein the virtual segments are distributed among physical memory resources. 24.根据权利要求23所述的方法,其中,在虚拟存储器卷上设置虚拟分段,并且其中,对于存储器客户机,该虚拟存储器卷表现为物理存储器资源。24. The method of claim 23, wherein virtual segmentation is set on a virtual memory volume, and wherein, to a memory client, the virtual memory volume appears as a physical memory resource. 25.根据权利要求22所述的方法,其中,该提供步骤包括:25. The method of claim 22, wherein the providing step comprises: 监测存储器客户机对虚拟存储器进行的存取;以及monitoring accesses to virtual storage by storage clients; and 根据该存取,将虚拟存储器资源动态地分配给物理存储器资源。Based on the access, virtual memory resources are dynamically allocated to physical memory resources. 26.根据权利要求25所述的方法,其中,物理存储器资源的特征由性能参数表示,并且其中,该动态分配步骤包括:26. The method of claim 25, wherein the physical memory resources are characterized by performance parameters, and wherein the dynamic allocation step comprises: 根据该性能参数以及存储器客户机进行存取的特性,将虚拟存储器资源分配给物理存储器资源。Based on the performance parameters and the nature of the accesses made by the memory clients, virtual memory resources are allocated to physical memory resources. 27.根据权利要求25所述的方法,其中,该动态分配步骤包括:27. The method of claim 25, wherein the dynamic allocation step comprises: 根据存储器客户机使用虚拟存储器的等级,将虚拟存储器资源分配给物理存储器资源。The virtual memory resources are allocated to the physical memory resources according to the level at which the virtual memory is used by the memory clients. 28.根据权利要求22所述的方法,其中,存储器客户机存取由多个虚拟存储器资源所保持的数据,并且该方法进一步包括:28. The method of claim 22, wherein the memory client accesses data held by a plurality of virtual memory resources, and the method further comprises: 测试保持该数据的多个虚拟存储器资源;以及testing the multiple virtual memory resources holding the data; and 根据该测试,识别存储器客户机能够从其存取该数据的一组最佳虚拟存储器资源。From this test, an optimal set of virtual memory resources from which the memory client can access the data is identified. 29.根据权利要求28所述的方法,该方法进一步包括:29. The method of claim 28, further comprising: 在存储器客户机将从其存取该数据的组中选择虚拟存储器资源。A virtual memory resource is selected among the groups from which the memory client will access the data. 30.根据权利要求22所述的方法,该方法进一步包括:30. The method of claim 22, further comprising: 识别网络上的新存储器节点,该新存储器节点与新物理存储器资源相关;以及identifying a new memory node on the network that is associated with a new physical memory resource; and 对新物理存储器资源分配一部分虚拟存储器资源。A portion of the virtual memory resource is allocated to the new physical memory resource.
CNA028247108A 2001-12-10 2002-12-04 Manage memory resources attached to data networks Pending CN1602480A (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
IL147073 2001-12-10
IL14707301A IL147073A0 (en) 2001-12-10 2001-12-10 Method for managing the storage resources attached to a data network
US10/279,755 US20030110263A1 (en) 2001-12-10 2002-10-23 Managing storage resources attached to a data network
US10/279,755 2002-10-23

Publications (1)

Publication Number Publication Date
CN1602480A true CN1602480A (en) 2005-03-30

Family

ID=26324055

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA028247108A Pending CN1602480A (en) 2001-12-10 2002-12-04 Manage memory resources attached to data networks

Country Status (5)

Country Link
EP (1) EP1456766A1 (en)
JP (1) JP2005512232A (en)
CN (1) CN1602480A (en)
CA (1) CA2469624A1 (en)
WO (1) WO2003050707A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101834904A (en) * 2010-05-14 2010-09-15 杭州华三通信技术有限公司 Method and equipment for database backup
WO2013004136A1 (en) * 2011-07-01 2013-01-10 华为数字技术(成都)有限公司 Distributed storage method, device and system
CN103186349A (en) * 2011-12-27 2013-07-03 杭州信核数据科技有限公司 Block-level distributed storage system and data reading-writing method thereof
TWI413376B (en) * 2008-05-02 2013-10-21 Hon Hai Prec Ind Co Ltd Apparatus and method for managing network memory
WO2014101218A1 (en) * 2012-12-31 2014-07-03 华为技术有限公司 Computing and storage integrated cluster system
CN104067288A (en) * 2012-01-23 2014-09-24 西里克斯系统公司 storage encryption
CN105657057A (en) * 2012-12-31 2016-06-08 华为技术有限公司 Calculation and storage fused cluster system
CN106105152A (en) * 2014-05-22 2016-11-09 华为技术有限公司 A node interconnection device, resource control node and server system
WO2019219059A1 (en) * 2018-05-17 2019-11-21 杭州海康威视系统技术有限公司 Method, apparatus and system for storing data, and method for reading data, apparatus, and system

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1559056A4 (en) 2002-10-25 2006-08-23 Symbol Technologies Inc Optimization of a binary tree traversal with secure communications
US7480658B2 (en) 2003-07-16 2009-01-20 Joltid Ltd. Distributed database system and method having nodes co-ordinated in a decentralized manner
EP1668486A2 (en) 2003-08-14 2006-06-14 Compellent Technologies Virtual disk drive system and method
US9489150B2 (en) 2003-08-14 2016-11-08 Dell International L.L.C. System and method for transferring data between different raid data storage types for current data and replay data
CN100440888C (en) * 2004-01-17 2008-12-03 中国科学院计算技术研究所 Management system and method for large-scale service system based on network storage and resource virtualization
CN100440830C (en) * 2004-04-13 2008-12-03 中国科学院计算技术研究所 System and method for dynamically reconfigurable network-based computing environment
US7712102B2 (en) * 2004-07-30 2010-05-04 Hewlett-Packard Development Company, L.P. System and method for dynamically configuring a plurality of load balancers in response to the analyzed performance data
GB2453556A (en) * 2007-10-10 2009-04-15 Ericsson Telefon Ab L M Dynamic allocation of E1 link circuits to a pool of MSC-S and ISUP identifier translation by a Media Gateway
JP2009238114A (en) * 2008-03-28 2009-10-15 Hitachi Ltd Storage management method, storage management program, storage management apparatus, and storage management system
JP5585235B2 (en) 2010-06-21 2014-09-10 富士通株式会社 Storage control device, storage control method, storage system
US9098212B2 (en) 2011-04-26 2015-08-04 Hitachi, Ltd. Computer system with storage apparatuses including physical and virtual logical storage areas and control method of the computer system
WO2016033691A1 (en) * 2014-09-04 2016-03-10 Iofabric Inc. Application centric distributed storage system and method
JP6375849B2 (en) 2014-10-09 2018-08-22 富士通株式会社 FILE SYSTEM, MANAGEMENT DEVICE CONTROL PROGRAM, AND FILE SYSTEM CONTROL METHOD

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6636242B2 (en) * 1999-08-31 2003-10-21 Accenture Llp View configurer in a presentation services patterns environment
US6970939B2 (en) * 2000-10-26 2005-11-29 Intel Corporation Method and apparatus for large payload distribution in a network
US20030033398A1 (en) * 2001-08-10 2003-02-13 Sun Microsystems, Inc. Method, system, and program for generating and using configuration policies

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI413376B (en) * 2008-05-02 2013-10-21 Hon Hai Prec Ind Co Ltd Apparatus and method for managing network memory
CN101834904A (en) * 2010-05-14 2010-09-15 杭州华三通信技术有限公司 Method and equipment for database backup
WO2013004136A1 (en) * 2011-07-01 2013-01-10 华为数字技术(成都)有限公司 Distributed storage method, device and system
CN103186349A (en) * 2011-12-27 2013-07-03 杭州信核数据科技有限公司 Block-level distributed storage system and data reading-writing method thereof
CN103186349B (en) * 2011-12-27 2016-03-02 杭州信核数据科技股份有限公司 Block level distributed memory system and data read-write method thereof
CN104067288B (en) * 2012-01-23 2017-03-29 西里克斯系统公司 Storage encryption method
CN104067288A (en) * 2012-01-23 2014-09-24 西里克斯系统公司 storage encryption
US9733848B2 (en) 2012-12-31 2017-08-15 Huawei Technologies Co., Ltd. Method and system for pooling, partitioning, and sharing network storage resources
CN105657057A (en) * 2012-12-31 2016-06-08 华为技术有限公司 Calculation and storage fused cluster system
WO2014101218A1 (en) * 2012-12-31 2014-07-03 华为技术有限公司 Computing and storage integrated cluster system
US10082972B2 (en) 2012-12-31 2018-09-25 Huawei Technologies Co., Ltd. Method and system for pooling, partitioning, and sharing network storage resources
US10481804B2 (en) 2012-12-31 2019-11-19 Huawei Technologies Co., Ltd. Cluster system with calculation and storage converged
US11042311B2 (en) 2012-12-31 2021-06-22 Huawei Technologies Co., Ltd. Cluster system with calculation and storage converged
CN106105152A (en) * 2014-05-22 2016-11-09 华为技术有限公司 A node interconnection device, resource control node and server system
CN106105152B (en) * 2014-05-22 2018-07-13 华为技术有限公司 A node interconnection device and server system
US10310756B2 (en) 2014-05-22 2019-06-04 Huawei Technologies Co., Ltd. Node interconnection apparatus, resource control node, and server system
US11023143B2 (en) 2014-05-22 2021-06-01 Huawei Technologies Co., Ltd. Node interconnection apparatus, resource control node, and server system
US11789619B2 (en) 2014-05-22 2023-10-17 Huawei Technologies Co., Ltd. Node interconnection apparatus, resource control node, and server system
US11899943B2 (en) 2014-05-22 2024-02-13 Huawei Technologies Co., Ltd. Node interconnection apparatus, resource control node, and server system
WO2019219059A1 (en) * 2018-05-17 2019-11-21 杭州海康威视系统技术有限公司 Method, apparatus and system for storing data, and method for reading data, apparatus, and system

Also Published As

Publication number Publication date
CA2469624A1 (en) 2003-06-19
WO2003050707A8 (en) 2004-11-04
EP1456766A1 (en) 2004-09-15
WO2003050707A1 (en) 2003-06-19
JP2005512232A (en) 2005-04-28

Similar Documents

Publication Publication Date Title
CN1602480A (en) Manage memory resources attached to data networks
US20030110263A1 (en) Managing storage resources attached to a data network
EP3230868B1 (en) Multiple transaction logs in a distributed storage system
US12204496B2 (en) Metadata control in a load-balanced distributed storage system
JP3944175B2 (en) Dynamic processor reallocation between partitions in a computer system.
US9569244B2 (en) Implementing dynamic adjustment of I/O bandwidth for virtual machines using a single root I/O virtualization (SRIOV) adapter
CN1186730C (en) System and method for managing space distribution in storage equipment of caculation environment
US10895995B2 (en) Capacity based load balancing in distributed storage systems with deduplication and compression functionalities
JP5104855B2 (en) Load distribution program, load distribution method, and storage management apparatus
JP2016528617A (en) Resource load balancing
JP2005056077A (en) Database control method
US12229085B2 (en) Access redirection in a distributive file system
US11513849B2 (en) Weighted resource cost matrix scheduler
US20160364145A1 (en) System and Method for Managing a Non-Volatile Storage Resource as a Shared Resource in a Distributed System
EP1949230A2 (en) Method and apparatus for increasing throughput in a storage server
US11144445B1 (en) Use of compression domains that are more granular than storage allocation units
US12079477B2 (en) Optimizing backend workload processing in a storage system
US20240012698A1 (en) Method and Apparatus for Optimizing System Call (Syscall) Processing
AU2002348882A1 (en) Managing storage resources attached to a data network

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication