CN106502721B - A command unloading method, device and physical machine - Google Patents
A command unloading method, device and physical machine Download PDFInfo
- Publication number
- CN106502721B CN106502721B CN201610853875.5A CN201610853875A CN106502721B CN 106502721 B CN106502721 B CN 106502721B CN 201610853875 A CN201610853875 A CN 201610853875A CN 106502721 B CN106502721 B CN 106502721B
- Authority
- CN
- China
- Prior art keywords
- data
- operated
- virtual
- host
- source address
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44594—Unloading
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明公开了一种命令卸载方法、装置及物理机,涉及通信技术领域,能够在处理操作指令(如拷贝或者清零指令)时,有效减少对物理机的硬件资源的浪费。本发明的方法包括:虚拟机截获应用程序下发的操作指令,操作指令用于指示对待操作数据执行指定操作;虚拟机根据操作指令,生成卸载命令,并向主机发送卸载命令,卸载命令包括拷贝卸载命令或清零卸载命令;主机将卸载命令中的虚拟源地址转换为待操作数据的物理源地址,虚拟源地址用于表示待操作数据在虚拟机的虚拟磁盘中的位置;主机根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作。本发明适用于命令卸载过程。
The invention discloses a command unloading method, device and physical machine, relates to the technical field of communication, and can effectively reduce the waste of hardware resources of the physical machine when processing operation instructions (such as copying or clearing instructions). The method of the present invention includes: the virtual machine intercepts the operation instruction issued by the application program, and the operation instruction is used to instruct to perform a specified operation on the data to be operated; the virtual machine generates an unload command according to the operation instruction, and sends the unload command to the host, and the unload command includes copying Unload command or clear unload command; the host converts the virtual source address in the unload command into the physical source address of the data to be operated, and the virtual source address is used to indicate the location of the data to be operated in the virtual disk of the virtual machine; The virtual destination address in and the physical source address of the data to be operated perform the specified operation on the data to be operated. The present invention is applicable to command offloading process.
Description
技术领域technical field
本发明涉及通信技术领域,尤其涉及一种命令卸载方法、装置及物理机。The present invention relates to the field of communication technology, in particular to a command offloading method, device and physical machine.
背景技术Background technique
目前,无论是为了实现虚拟机之间的拷贝卸载或是虚拟机自身的清零命令卸载过程,都需要虚拟机在接收到应用程序下发的操作指令之后,将操作指令经由IO(英文:Input/Output,中文:输入/输出)环发送至后端进程,并由主机根据该操作指令生成相应的SCSI(英文:Small Computer System Interface,中文:小型计算机系统接口)指令,之后发送至诸如IP SAN(英文:IP based SAN,中文:基于IP的SAN设备)等物理存储设备,由物理存储设备解析并执行该SCSI指令,从而在物理存储设备上完成数据的拷贝或是清零操作。At present, whether it is to realize the unloading process of copying between virtual machines or the clearing command unloading process of the virtual machine itself, it is necessary for the virtual machine to send the operation command through the IO (English: Input /Output, Chinese: Input/Output) ring is sent to the back-end process, and the host computer generates the corresponding SCSI (English: Small Computer System Interface, Chinese: Small Computer System Interface) command according to the operation command, and then sends it to such as IP SAN (English: IP based SAN, Chinese: IP-based SAN device) and other physical storage devices, the physical storage device parses and executes the SCSI command, thereby completing data copy or clearing operations on the physical storage device.
由此可见,上述操作在执行过程中,需要将生成的SCSI指令最终发送至对应的物理存储设备,之后由该物理存储设备来执行相应的操作。然而,考虑到上述数据的拷贝或是清零操作都需要由物理机的物理层硬件设备来完成,甚至需要物理机的CPU(英文:CentralProcessing Unit,中文:中央处理器)和网络资源的参与,对物理机的硬件资源要求较高,也带来了对物理机硬件资源和网络资源使用上的浪费。It can be seen that, during the execution of the above operations, the generated SCSI command needs to be finally sent to the corresponding physical storage device, and then the physical storage device performs the corresponding operation. However, considering that the copying or clearing of the above data needs to be completed by the physical layer hardware device of the physical machine, and even requires the participation of the CPU (English: Central Processing Unit, Chinese: central processing unit) and network resources of the physical machine, High requirements on the hardware resources of the physical machine also lead to a waste of the hardware resources of the physical machine and the use of network resources.
发明内容Contents of the invention
本发明提供一种命令卸载方法、装置及物理机,能够减少在处理操作指令(如拷贝或者清零指令)时,对物理机的硬件资源的浪费。The invention provides a command unloading method, device and physical machine, which can reduce the waste of hardware resources of the physical machine when processing operation instructions (such as copying or clearing instructions).
为达到上述目的,本发明采用如下技术方案:To achieve the above object, the present invention adopts the following technical solutions:
第一方面,本发明提供一种命令卸载方法,该方法用于物理机,物理机运行有主机Host,该主机上运行有虚拟机。该方法包括:虚拟机截获应用程序下发的操作指令,该操作指令用于指示对待操作数据执行指定操作;虚拟机根据操作指令,生成卸载命令,并向主机发送卸载命令,该卸载命令包括拷贝卸载命令或清零卸载命令;主机将卸载命令中的虚拟源地址转换为待操作数据的物理源地址,该虚拟源地址用于表示待操作数据在虚拟机的虚拟磁盘中的位置;主机根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作。由此可见,这样虽然需要虚拟机与主机共同来完成指定操作的执行过程,但由于主机是根据虚拟机生成的卸载命令来完成指定操作的,因此,无论当前应用本发明的系统为集群文件系统、分布式共享存储系统,或是其他系统,主机均能够识别该卸载命令。并且,主机和虚拟机均属于物理机的一部分,在上述执行过程中,主要由主机在接收到卸载命令之后执行相应操作,而无需物理存储设备执行相应的操作,因此,采用本发明所提供的实现方式,能够在处理操作指令(如拷贝或者清零指令)时,有效减少对物理机的硬件资源的浪费。In a first aspect, the present invention provides a command unloading method, the method is used for a physical machine, the physical machine runs a host, and the host runs a virtual machine. The method includes: the virtual machine intercepts the operation instruction issued by the application program, and the operation instruction is used to instruct to perform a specified operation on the data to be operated; the virtual machine generates an unload command according to the operation instruction, and sends the unload command to the host, and the unload command includes Unload command or clear unload command; the host converts the virtual source address in the unload command into the physical source address of the data to be operated, and the virtual source address is used to represent the location of the data to be operated in the virtual disk of the virtual machine; The virtual destination address in the command and the physical source address of the data to be operated perform the specified operation on the data to be operated. It can be seen that although the virtual machine and the host computer are required to complete the execution process of the specified operation together, since the host computer completes the specified operation according to the uninstall command generated by the virtual machine, no matter whether the system currently applying the present invention is a cluster file system , a distributed shared storage system, or other systems, the host can recognize the uninstall command. Moreover, both the host and the virtual machine are part of the physical machine. In the above execution process, the host performs corresponding operations after receiving the unloading command, without the need for a physical storage device to perform corresponding operations. Therefore, using the The implementation manner can effectively reduce the waste of hardware resources of the physical machine when processing operation instructions (such as copy or clear instructions).
在一种可能的设计中,当指定操作为拷贝操作时,待操作数据包括至少一个数据块,虚拟机可以生成至少一个数据块中每个数据块的拷贝卸载命令,并向主机发送每个数据块的拷贝卸载命令。需要说明的是,虚拟机可以同时生成每个数据块的拷贝卸载命令,并打包向主机发送,或是每次只生成一个数据块的拷贝卸载命令,并向主机发送,之后当主机执行完该拷贝卸载命令指示的拷贝操作之后,再生成下一条拷贝卸载命令,供主机执行,直至待操作数据中的所有数据块都完成拷贝操作。In a possible design, when the specified operation is a copy operation, the data to be operated includes at least one data block, and the virtual machine can generate a copy unload command for each data block in the at least one data block, and send each data block to the host Block copy unload command. It should be noted that the virtual machine can generate a copy and unload command for each data block at the same time, and package and send it to the host, or generate a copy and unload command for only one data block at a time, and send it to the host, and then when the host finishes executing the command After the copy operation indicated by the copy unload command, the next copy unload command is generated for execution by the host until all data blocks in the data to be operated are copied.
在一种可能的设计中,在虚拟机截获应用程序下发的操作指令之后,虚拟机还需要向主机发送创建指令,该创建指令中携带有每个数据块的拷贝卸载命令中数据块的长度,创建指令用于在目的虚拟机的磁盘文件系统中创建空文件,目的虚拟机运行在虚拟机运行的主机上,或运行在除主机以外的主机上;虚拟机接收目的虚拟机反馈的虚拟目的地址,该虚拟目的地址是空文件中分配给待操作数据的目的数据块的虚拟地址,目的数据块的长度大于或等于每个数据块的拷贝卸载命令中数据块的长度。这样一来,就可以确保主机在执行拷贝操作时,目的虚拟机中已经为这次拷贝过程预留出一定空间,即上述创建的空文件。In a possible design, after the virtual machine intercepts the operation command issued by the application, the virtual machine also needs to send the creation command to the host, and the creation command carries the length of the data block in the copy unload command of each data block , the creation instruction is used to create an empty file in the disk file system of the target virtual machine. The target virtual machine runs on the host where the virtual machine is running, or runs on a host other than the host; the virtual machine receives the feedback of the virtual target from the target virtual machine address, the virtual destination address is the virtual address allocated to the destination data block of the data to be operated in the empty file, and the length of the destination data block is greater than or equal to the length of the data block in the copy unload command of each data block. In this way, it can be ensured that when the host performs the copy operation, a certain space has been reserved for the copy process in the destination virtual machine, that is, the empty file created above.
在一种可能的设计中,该方法应用于分布式共享存储系统,则主机对待操作数据执行指定操作,可以具体实现为:主机将待操作数据的虚拟目的地址更新为待操作数据的物理源地址;主机根据更新的虚拟目的地址,将待操作数据的物理源地址对应的数据块的引用计数的数值加一,该引用计数的值用于表示待操作数据的物理地址对应的数据块被引用的次数。由此可见,采用上述方式来实现待操作数据的拷贝操作,仅仅建立了地址之间的映射关系,并增加待操作数据的物理源地址的引用计数。因此,对于虚拟机而言,降低了虚拟机存储操作对虚拟机计算网络资源的消耗,同时,通过增加引用计数还起到了加速的效果。In a possible design, the method is applied to a distributed shared storage system, and the host performs specified operations on the data to be operated, which can be specifically implemented as: the host updates the virtual destination address of the data to be operated to the physical source address of the data to be operated ; According to the updated virtual destination address, the host adds one to the value of the reference count of the data block corresponding to the physical source address of the data to be operated, and the value of the reference count is used to indicate that the data block corresponding to the physical address of the data to be operated is referenced frequency. It can be seen that the copy operation of the data to be operated using the above method only establishes the mapping relationship between addresses and increases the reference count of the physical source address of the data to be operated. Therefore, for the virtual machine, the consumption of the computing network resources of the virtual machine by the storage operation of the virtual machine is reduced, and at the same time, an acceleration effect is also achieved by increasing the reference count.
在一种可能的设计中,该方法应用于集群文件系统,则主机对待操作数据执行指定操作,可以具体实现为:主机建立待操作数据的虚拟目的地址与待操作数据的物理源地址之间的映射关系;主机将待操作数据的物理源地址对应的数据块拷贝到待操作数据的物理目的地址;主机将映射关系中的待操作数据的物理源地址替换为待操作数据的物理目的地址。由此可见,采用上述方式来实现待操作数据的拷贝操作,需要先将数据块从物理源地址拷贝到物理目的地址,之后建立虚拟源地址与物理目的地址之间的映射关系。对于虚拟机而言,降低了虚拟机存储操作对虚拟机计算网络资源的消耗,但相比较于前一种可能的设计,则无法达到有效的拷贝加速的效果。In a possible design, the method is applied to the cluster file system, and the host performs specified operations on the data to be operated, which can be implemented as follows: the host establishes a link between the virtual destination address of the data to be operated and the physical source address of the data to be operated Mapping relationship: the host copies the data block corresponding to the physical source address of the data to be operated to the physical destination address of the data to be operated; the host replaces the physical source address of the data to be operated in the mapping relationship with the physical destination address of the data to be operated. It can be seen that to implement the copy operation of the data to be operated in the above manner, it is necessary to first copy the data block from the physical source address to the physical destination address, and then establish a mapping relationship between the virtual source address and the physical destination address. For the virtual machine, the consumption of virtual machine computing network resources by the virtual machine storage operation is reduced, but compared with the former possible design, the effect of effective copy acceleration cannot be achieved.
在一种可能的设计中,主机将卸载命令中的虚拟源地址转换为待操作数据的物理源地址的过程,可以具体实现为:在待操作数据的存储方式为块存储的情况下,主机将待操作数据的虚拟源地址转换为待操作数据的物理源地址;或者,在待操作数据的存储方式为对象存储的情况下,主机将待操作数据的虚拟源地址转换为对象标识,对象标识为待操作数据对应的数据块的标识。由此可见,对于不同的存储方式,需要采用不同的地址转换方式。In a possible design, the process of the host converting the virtual source address in the unload command into the physical source address of the data to be operated can be implemented as follows: when the data to be operated is stored in block storage, the host will The virtual source address of the data to be operated is converted into the physical source address of the data to be operated; or, when the storage method of the data to be operated is object storage, the host converts the virtual source address of the data to be operated into an object identifier, and the object identifier is The identifier of the data block corresponding to the data to be operated. It can be seen that for different storage methods, different address conversion methods need to be adopted.
在一种可能的设计中,当卸载命令为清零卸载命令时,主机根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作的过程,可以具体实现为:当虚拟磁盘为待操作数据模拟的虚拟磁盘时,主机将待操作数据的物理源地址的比特图中每一位清零;或者,当虚拟磁盘为分布式共享存储系统模拟的虚拟磁盘时,主机将待操作数据的对象标识释放,并释放待操作数据对应的数据块;或者,当虚拟磁盘为集群文件系统模拟的虚拟磁盘时,主机将待操作数据的数据块释放;或者,当不存在待操作数据的虚拟源地址与待操作数据的物理源地址之间的映射关系时,主机将待操作数据的物理源地址对应的数据块写零。由此可见,在本发明中,需要根据虚拟磁盘的模拟情况,来选择合适的清零方案。In a possible design, when the unload command is a clear unload command, the host performs a specified operation on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated, which can be specifically implemented as: When the virtual disk is simulated by the data to be operated, the host will clear each bit in the bitmap of the physical source address of the data to be operated; or, when the virtual disk is simulated by the distributed shared storage system, the host will Release the object identifier of the data to be operated, and release the data block corresponding to the data to be operated; or, when the virtual disk is a virtual disk simulated by the cluster file system, the host will release the data block of the data to be operated; or, when there is no When operating the mapping relationship between the virtual source address of the data and the physical source address of the data to be operated, the host writes zeros to the data block corresponding to the physical source address of the data to be operated. It can be seen that, in the present invention, it is necessary to select an appropriate zeroing scheme according to the simulation situation of the virtual disk.
第二方面,本发明提供一种命令卸载装置,该命令卸载装置包括主机Host和虚拟机。虚拟机,用于截获应用程序下发的操作指令,操作指令用于指示对待操作数据执行指定操作;虚拟机,还用于根据操作指令,生成卸载命令,并向主机发送卸载命令,卸载命令包括拷贝卸载命令或清零卸载命令;主机,用于将卸载命令中的虚拟源地址转换为待操作数据的物理源地址,虚拟源地址用于表示待操作数据在虚拟机的虚拟磁盘中的位置;主机,还用于根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作。上述命令卸载装置可以用于完成上述方法示例中主机和虚拟机所执行的功能,该功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。其中,该硬件或是软件包括至少一个上述功能相应的模块。需要说明的是,上述命令卸载装置中的主机和虚拟机还能用于实现第一方面所述的各种功能。In a second aspect, the present invention provides a command unloading device, which includes a host Host and a virtual machine. The virtual machine is used to intercept the operation instruction issued by the application program, and the operation instruction is used to instruct to perform a specified operation on the data to be operated; the virtual machine is also used to generate an uninstall command according to the operation instruction, and send the uninstall command to the host. The uninstall command includes Copying the unloading command or clearing the unloading command; the host is used to convert the virtual source address in the unloading command into the physical source address of the data to be operated, and the virtual source address is used to indicate the position of the data to be operated in the virtual disk of the virtual machine; The host computer is also used to perform specified operations on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated. The above command offloading device can be used to complete the functions executed by the host and the virtual machine in the above method example, and the functions can be realized by hardware, or by executing corresponding software by hardware. Wherein, the hardware or software includes at least one module with corresponding functions above. It should be noted that the host computer and the virtual machine in the above-mentioned command offloading device can also be used to implement various functions described in the first aspect.
第三方面,本发明提供一种物理机。该物理机包括硬件层、运行在硬件层之上的主机Host和虚拟机。虚拟机,用于截获应用程序下发的操作指令,操作指令用于指示对待操作数据执行指定操作;虚拟机,还用于根据操作指令,生成卸载命令,并向主机发送卸载命令,卸载命令包括拷贝卸载命令或清零卸载命令;主机,用于将卸载命令中的虚拟源地址转换为待操作数据的物理源地址,虚拟源地址用于表示待操作数据在虚拟机的虚拟磁盘中的位置;主机,还用于根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作。In a third aspect, the present invention provides a physical machine. The physical machine includes a hardware layer, a host running on the hardware layer, and a virtual machine. The virtual machine is used to intercept the operation instruction issued by the application program, and the operation instruction is used to instruct to perform a specified operation on the data to be operated; the virtual machine is also used to generate an uninstall command according to the operation instruction, and send the uninstall command to the host. The uninstall command includes Copying the unloading command or clearing the unloading command; the host is used to convert the virtual source address in the unloading command into the physical source address of the data to be operated, and the virtual source address is used to indicate the position of the data to be operated in the virtual disk of the virtual machine; The host computer is also used to perform specified operations on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated.
另外,该物理机中还可以包括处理器、输入设备、输出设备。该处理器被配置为支持该物理机执行上述方法中相应的功能。该输入设备和输出设备可以被视为通信接口,用于支持该物理机与其他设备之间的通信。需要说明的是,该物理机还可以包括存储器,该存储器用于与处理器耦合,具体可以包括用于实现第一方面所述的各种功能的虚拟机和主机,此外还能保存该物理机必要的程序指令和数据。In addition, the physical machine may further include a processor, an input device, and an output device. The processor is configured to support the physical machine to execute corresponding functions in the above method. The input device and the output device can be regarded as communication interfaces for supporting communication between the physical machine and other devices. It should be noted that the physical machine may also include a memory, which is used to couple with the processor, and specifically may include a virtual machine and a host for implementing various functions described in the first aspect, and may also save the physical machine Necessary program instructions and data.
本发明提供的命令卸载方法、装置及物理机,相比较于现有技术中借助物理存储设备来执行指定操作,在本发明中,虚拟机可以根据应用程序所下发的操作指令来生成卸载命令,之后由虚拟机将卸载命令向主机发送;主机在接收到虚拟机发送的卸载命令之后,将卸载命令所携带的虚拟源地址转换为待操作数据的物理源地址,之后,主机可以根据卸载命令所携带的虚拟目的地址和上述待操作数据的物理源地址,对待操作数据执行指定操作。这样虽然需要虚拟机与主机共同来完成指定操作的执行过程,但由于主机是根据虚拟机生成的卸载命令来完成指定操作的,因此,无论当前应用本发明的系统为集群文件系统、分布式共享存储系统,或是其他系统,主机均能够识别该卸载命令。并且,主机和虚拟机均属于物理机的一部分,在上述执行过程中,主要由主机在接收到卸载命令之后执行相应操作,而无需物理存储设备执行相应的操作,因此,采用本发明所提供的实现方式,能够在处理操作指令(如拷贝或者清零指令)时,有效减少对物理机的硬件资源的浪费。The command unloading method, device and physical machine provided by the present invention, compared with the prior art using physical storage devices to perform specified operations, in the present invention, the virtual machine can generate unloading commands according to the operating instructions issued by the application program , and then the virtual machine sends the unloading command to the host; after the host receives the unloading command sent by the virtual machine, it converts the virtual source address carried by the unloading command into the physical source address of the data to be operated, and then the host can follow the unloading command The carried virtual destination address and the physical source address of the data to be operated are used to perform specified operations on the data to be operated. Although the execution process of the specified operation needs to be completed by the virtual machine and the host computer together like this, because the host computer completes the specified operation according to the unloading command generated by the virtual machine, no matter whether the system currently applying the present invention is a cluster file system, distributed shared The storage system, or other systems, and the host can recognize the uninstall command. Moreover, both the host and the virtual machine are part of the physical machine. In the above execution process, the host performs corresponding operations after receiving the unloading command, without the need for a physical storage device to perform corresponding operations. Therefore, using the The implementation manner can effectively reduce the waste of hardware resources of the physical machine when processing operation instructions (such as copy or clear instructions).
附图说明Description of drawings
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其它的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.
图1为本发明实施例提供的一种数据中心结构示意图;FIG. 1 is a schematic structural diagram of a data center provided by an embodiment of the present invention;
图2为本发明实施例提供的另一种数据中心结构示意图;FIG. 2 is a schematic structural diagram of another data center provided by an embodiment of the present invention;
图3为本发明实施例提供的一种虚拟机和主机实现交互过程的示意图;FIG. 3 is a schematic diagram of an interaction process between a virtual machine and a host provided by an embodiment of the present invention;
图4为本发明实施例提供的一种命令卸载方法交互图;FIG. 4 is an interaction diagram of a command uninstallation method provided by an embodiment of the present invention;
图5为本发明实施例提供的另一种命令卸载方法交互图;FIG. 5 is an interaction diagram of another command uninstallation method provided by an embodiment of the present invention;
图6为本发明实施例提供的另一种命令卸载方法交互图;FIG. 6 is an interaction diagram of another command uninstallation method provided by an embodiment of the present invention;
图7为本发明实施例提供的一种命令卸载装置的结构示意图;FIG. 7 is a schematic structural diagram of a command unloading device provided by an embodiment of the present invention;
图8为本发明实施例提供的一种物理机的结构示意图。FIG. 8 is a schematic structural diagram of a physical machine provided by an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例,都属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
本发明实施例可以应用于不同的数据中心结构,本发明实施例不做限制。比如:该数据中心结构可以具体为设置有集群文件系统的结构,或是Server San架构等。The embodiments of the present invention may be applied to different data center structures, and the embodiments of the present invention are not limited thereto. For example, the data center structure may specifically be a structure provided with a cluster file system, or a Server San structure, and the like.
如图1所示为一种设置有集群文件系统01的数据中心结构,该数据中心中可以包括多个主机,比如包括主机100、主机200和主机300。在每个主机内部均可以运行一个或多个虚拟机,比如在主机100上运行着虚拟机10和虚拟机11,其中,每个虚拟机可以包括至少一个虚拟磁盘,比如虚拟机10包括虚拟磁盘12,且每个虚拟磁盘的后端都会存在一个后端存储服务进程,比如后端进程14,以实现为对应的虚拟机提供服务。其中,后端进程可以是qemu进程,也可以是专门的后端进程,比如blkback、vhost等。后端进程会将经IO环传送的命令向集群文件系统01发送,并由集群文件系统01经过网络02向IP SAN03发送,之后由IPSAN03对该命令进行处理。As shown in FIG. 1 , a data center structure provided with a cluster file system 01 may include multiple hosts, such as a host 100 , a host 200 and a host 300 . One or more virtual machines can be run inside each host, for example, a virtual machine 10 and a virtual machine 11 are running on the host 100, wherein each virtual machine can include at least one virtual disk, for example, the virtual machine 10 includes a virtual disk 12, and there will be a back-end storage service process at the back end of each virtual disk, such as the back-end process 14, so as to provide services for the corresponding virtual machines. Among them, the backend process can be a qemu process, or a special backend process, such as blkback, vhost, etc. The back-end process will send the command transmitted through the IO ring to the cluster file system 01, and the cluster file system 01 will send it to the IP SAN03 through the network 02, and then the command will be processed by the IPSAN03.
如图2所示为一种设置有分布式共享存储系统的Server San架构,该数据中心同样也可以包括多个主机,且每个主机内部均可以运行一个或多个虚拟机,虚拟机内部结构与如图1所示架构中的虚拟机类似,在此不做赘述。与图1的区别在于,后端进程会将经IO环传送的命令向分布式共享存储系统04发送,之后由分布式共享存储系统04将命令转发给对应的本地磁盘,并由本地磁盘对该命令进行处理。需要说明的是,上述本地磁盘可以为传送该命令的后端进程所对应的虚拟机所在主机内部的本地磁盘,也可以为设置在其他主机上的虚拟磁盘。比如主机400中,虚拟机40最终通过后端进程44将命令向分布式共享存储系统04发送,分布式共享系统04根据该命令本身的含义来确定是将该命令向本地磁盘46、本地磁盘47发送,或是向位于主机500或主机600中的本地磁盘发送。需要说明的是,本地磁盘的数量与主机内部所设置的虚拟机的数量无关,可以根据自身需求进行配置,比如主机500中虽然包括虚拟机50和虚拟机51,但在主机500中仅包括本地磁盘56,同样的,主机600中虽然仅包括虚拟机60,但可以在主机600中设置多个本地磁盘,即本地磁盘63和本地磁盘64。As shown in Figure 2, it is a Server San architecture with a distributed shared storage system. The data center can also include multiple hosts, and each host can run one or more virtual machines. The internal structure of the virtual machine Similar to the virtual machine in the architecture shown in FIG. 1 , details are not described here. The difference from Figure 1 is that the backend process will send the command transmitted through the IO ring to the distributed shared storage system 04, and then the distributed shared storage system 04 will forward the command to the corresponding local disk, and the local disk will command is processed. It should be noted that the above local disk may be a local disk inside the host where the virtual machine corresponding to the backend process transmitting the command resides, or may be a virtual disk set on another host. For example, in the host computer 400, the virtual machine 40 finally sends the command to the distributed shared storage system 04 through the back-end process 44, and the distributed shared storage system 04 determines whether to send the command to the local disk 46 or the local disk 47 according to the meaning of the command itself. or send to a local disk located in host 500 or host 600. It should be noted that the number of local disks has nothing to do with the number of virtual machines set inside the host, and can be configured according to its own needs. For example, although the host 500 includes virtual machines 50 and 51, the host 500 only includes the local Similarly, although the host 600 only includes the virtual machine 60 , multiple local disks can be set in the host 600 , that is, a local disk 63 and a local disk 64 .
本发明实施例提供一种命令卸载方法,可以应用于如图3所示的虚拟机700a,且该虚拟机700a可以位于上述两种数据中心结构中,但不仅限位于上述两种数据中心结构中。在本发明实施例中,在虚拟机700a内部,可以基于文件系统73posix接口来实现操作指令的捕获,也就是当应用程序下发拷贝、清零等操作指令时,虚拟机700a可以将该操作指令捕获,并生成相应的卸载命令,并传递给前端进程71。之后由前端进程71将卸载命令放到IO环72中传递给主机700b内部的后端进程73,并由后端进程73解析该卸载命令,最终交给分布式共享存储系统04或集群文件系统01进行处理。具体实现方式会在后文提出,在此不做赘述。The embodiment of the present invention provides a command offloading method, which can be applied to the virtual machine 700a shown in Figure 3, and the virtual machine 700a can be located in the above two data center structures, but not limited to the above two data center structures . In the embodiment of the present invention, within the virtual machine 700a, the capture of operation instructions can be realized based on the file system 73posix interface, that is, when the application program issues operation instructions such as copying and clearing, the virtual machine 700a can use the operation instruction capture, and generate a corresponding uninstall command, and pass it to the front-end process 71. Afterwards, the front-end process 71 puts the unload command into the IO ring 72 and passes it to the back-end process 73 inside the host 700b, and the back-end process 73 parses the unload command, and finally delivers it to the distributed shared storage system 04 or the cluster file system 01 to process. The specific implementation manner will be proposed later, and will not be repeated here.
以上述两种数据中心结构为例,如图4所示,该方法可以应用于物理机,该物理机上可以运行有主机(英文:Host),该主机上还可以运行有虚拟机,本发明可以由该虚拟机与该主机共同来实现,该方法流程包括:Taking the above two data center structures as an example, as shown in Figure 4, the method can be applied to a physical machine, a host (English: Host) can run on the physical machine, and a virtual machine can also run on the host, the present invention can It is realized jointly by the virtual machine and the host, and the process of the method includes:
101、虚拟机截获应用程序下发的操作指令。101. The virtual machine intercepts the operation instruction issued by the application program.
其中,操作指令用于指示对待操作数据执行指定操作。Wherein, the operation instruction is used to instruct to perform a specified operation on the data to be operated.
虚拟机的应用程序所下发的操作指令可以包括清零指令或是拷贝指令,比如虚拟机中的APP(英文:Application,中文:应用程序)对文件A中f偏移位置下发x长度的写0操作,或是源端虚拟机中的APP下发拷贝指令,将文件A从目录a拷贝到目的端虚拟机的目录b。需要说明的是,拷贝指令可以包括跨虚拟机拷贝指令,以及在同一虚拟机上进行拷贝操作的虚拟机内部拷贝指令中的至少一种。The operation instruction issued by the application program of the virtual machine may include a clear instruction or a copy instruction. For example, the APP (English: Application, Chinese: application program) in the virtual machine issues an x length Write 0 operation, or the APP in the source virtual machine sends a copy command to copy file A from directory a to directory b of the destination virtual machine. It should be noted that the copy instruction may include at least one of a cross-virtual machine copy instruction and a virtual machine internal copy instruction that performs a copy operation on the same virtual machine.
需要说明的是,在本发明中,虚拟机可以识别到应用程序下发的操作指令,并截获该操作指令。It should be noted that, in the present invention, the virtual machine can recognize the operation instruction issued by the application program, and intercept the operation instruction.
102、虚拟机根据操作指令,生成卸载命令。102. The virtual machine generates an uninstallation command according to the operation instruction.
其中,卸载命令包括拷贝卸载命令或清零卸载命令。Wherein, the uninstall command includes a copy uninstall command or a clear uninstall command.
需要说明的是,虚拟机可以根据所截获的操作指令,生成卸载命令,并由前端进程将生成的卸载命令通过IO环向后端进程发送。It should be noted that the virtual machine can generate an unload command according to the intercepted operation instruction, and the front-end process sends the generated unload command to the back-end process through the IO ring.
103、虚拟机向主机发送卸载命令。103. The virtual machine sends an uninstall command to the host.
104、主机将卸载命令中的虚拟源地址转换为待操作数据的物理源地址。104. The host converts the virtual source address in the unload command into a physical source address of the data to be operated.
其中,虚拟源地址用于表示待操作数据在虚拟机的虚拟磁盘中的位置。Wherein, the virtual source address is used to indicate the location of the data to be operated in the virtual disk of the virtual machine.
105、主机根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作。105. The host performs a specified operation on the data to be operated according to the virtual destination address in the offload command and the physical source address of the data to be operated.
在主机接收到虚拟机发送的卸载命令之后,解析该卸载命令,并按照该卸载命令所指示的操作类型来完成指定操作。如图3所示,主机700b中的后端进程73在接收到虚拟机700a发送的卸载命令之后,发送至分布式共享存储系统04或集群文件系统01,并根据不同的系统类型,采用相应的方式完成指定操作。After the host receives the uninstall command sent by the virtual machine, it parses the uninstall command, and completes the specified operation according to the operation type indicated by the uninstall command. As shown in FIG. 3, after receiving the unloading command sent by the virtual machine 700a, the backend process 73 in the host 700b sends it to the distributed shared storage system 04 or the cluster file system 01, and uses the corresponding way to complete the specified operation.
在本发明中,虚拟机可以根据应用程序所下发的操作指令来生成卸载命令,之后由虚拟机将卸载命令向主机发送;主机在接收到虚拟机发送的卸载命令之后,将卸载命令所携带的虚拟源地址转换为待操作数据的物理源地址,之后,主机可以根据卸载命令所携带的虚拟目的地址和上述待操作数据的物理源地址,对待操作数据执行指定操作。这样虽然需要虚拟机与主机共同来完成指定操作的执行过程,但由于主机是根据虚拟机生成的卸载命令来完成指定操作的,因此,无论当前应用本发明的系统为集群文件系统、分布式共享存储系统,或是其他系统,主机均能够识别该卸载命令。并且,主机和虚拟机均属于物理机的一部分,在上述执行过程中,主要由主机在接收到卸载命令之后执行相应操作,而无需物理存储设备执行相应的操作,因此,采用本发明所提供的实现方式,能够在处理操作指令(如拷贝或者清零指令)时,有效减少对物理机的硬件资源的浪费。In the present invention, the virtual machine can generate an uninstall command according to the operation instruction issued by the application program, and then the virtual machine sends the uninstall command to the host; after the host receives the uninstall command sent by the virtual machine, it sends the uninstall command The virtual source address of the data to be operated is converted into the physical source address of the data to be operated. After that, the host can perform specified operations on the data to be operated according to the virtual destination address carried in the offload command and the physical source address of the data to be operated. Although the execution process of the specified operation needs to be completed by the virtual machine and the host computer together like this, because the host computer completes the specified operation according to the unloading command generated by the virtual machine, no matter whether the system currently applying the present invention is a cluster file system, distributed shared The storage system, or other systems, and the host can recognize the uninstall command. Moreover, both the host and the virtual machine are part of the physical machine. In the above execution process, the host performs corresponding operations after receiving the unloading command, without the need for a physical storage device to perform corresponding operations. Therefore, using the The implementation manner can effectively reduce the waste of hardware resources of the physical machine when processing operation instructions (such as copy or clear instructions).
考虑到当指定操作为拷贝时,待操作数据所包括的数据块的个数为至少一个,为了确保能够将待操作数据完整的进行拷贝,在本发明实施例的一个实现方式中,需要分别生成每个数据块的拷贝卸载命令,从而使主机能够执行每个数据块的拷贝操作。因此,如图4所示的步骤102虚拟机根据操作指令,生成卸载命令,可以具体实现为步骤1021;如图4所示的步骤103虚拟机向主机发送卸载命令,可以具体实现为步骤1031:Considering that when the specified operation is copy, the number of data blocks included in the data to be operated is at least one, in order to ensure that the data to be operated can be completely copied, in an implementation of the embodiment of the present invention, it is necessary to generate The copy offload command for each data block, thereby enabling the host to perform a copy operation for each data block. Therefore, in step 102 shown in Figure 4, the virtual machine generates an uninstall command according to the operation instruction, which can be specifically implemented as step 1021; in step 103 shown in Figure 4, the virtual machine sends an uninstall command to the host, which can be specifically implemented as step 1031:
1021、虚拟机生成至少一个数据块中每个数据块的拷贝卸载命令。1021. The virtual machine generates a copy and unload command for each data block in at least one data block.
1031、虚拟机向主机发送每个数据块的拷贝卸载命令。1031. The virtual machine sends a copy and unload command for each data block to the host.
在本发明中,虚拟机可以同时将每个数据块的拷贝卸载命令一起向主机发送,或每次只向主机发送一个拷贝卸载命令,并当该主机执行完这一个拷贝卸载命令之后,再向该主机发送下一个拷贝卸载命令,直至该主机执行完所有拷贝卸载命令所指示的操作。In the present invention, the virtual machine can send the copy unload command of each data block to the host at the same time, or only send one copy unload command to the host at a time, and after the host finishes executing the copy unload command, send the copy unload command to the host. The host sends the next copy unload command until the host completes the operations indicated by all the copy unload commands.
需要说明的是,虚拟机可以按照生成数据块的先后顺序来生成每个数据块对应的拷贝卸载命令,并由主机依次执行每个数据块的拷贝操作,但在本发明中,并不限定虚拟机生成拷贝卸载命令的顺序,以及主机执行拷贝操作的顺序,只要可以确保待操作数据能够被全部被拷贝即可。It should be noted that the virtual machine can generate the copy and unload command corresponding to each data block according to the order in which the data blocks are generated, and the host will sequentially execute the copy operation of each data block, but in the present invention, the virtual machine is not limited to The order in which the host generates the copy unload command and the order in which the host executes the copy operation, as long as it can ensure that all the data to be operated can be copied.
另外,考虑到拷贝操作在执行过程中,需要在目的虚拟机中创建一个用于存储待操作数据的空文件,因此,在进行上述操作之前,需要先在目的虚拟机的磁盘文件系统中创建空文件。在执行完如图4所示的步骤101虚拟机获取应用程序下发的操作指令之后,还需要执行步骤106和步骤107:In addition, considering that during the execution of the copy operation, an empty file for storing the data to be operated needs to be created in the destination virtual machine. Therefore, before performing the above operations, it is necessary to create an empty file in the disk file system of the destination virtual machine. document. After executing step 101 shown in Figure 4, the virtual machine acquires the operating instructions issued by the application, and then steps 106 and 107 need to be performed:
106、虚拟机向主机发送创建指令。106. The virtual machine sends a creation instruction to the host.
其中,创建指令中携带有每个数据块的拷贝卸载命令中数据块的长度,创建指令用于在目的虚拟机的磁盘文件系统中创建空文件,目的虚拟机运行在虚拟机运行的主机上,或运行在除主机以外的主机上。Wherein, the creation instruction carries the length of the data block in the copy and unload command of each data block, and the creation instruction is used to create an empty file in the disk file system of the target virtual machine, and the target virtual machine runs on the host machine where the virtual machine runs, Or run on a host other than the host.
107、虚拟机接收目的虚拟机反馈的虚拟目的地址。107. The virtual machine receives the virtual destination address fed back by the destination virtual machine.
其中,虚拟目的地址是空文件中分配给待操作数据的目的数据块的虚拟地址,目的数据块的长度大于或等于每个数据块的拷贝卸载命令中数据块的长度。Wherein, the virtual destination address is the virtual address of the destination data block assigned to the data to be operated in the empty file, and the length of the destination data block is greater than or equal to the length of the data block in the copy unload command of each data block.
这样一来,就能够为待操作数据在目的虚拟机的磁盘文件系统中预留出一定的存储空间,以便于将待操作数据从虚拟源地址拷贝到目的源地址。In this way, a certain storage space can be reserved in the disk file system of the target virtual machine for the data to be operated, so that the data to be operated can be copied from the virtual source address to the target source address.
以如图5所示的拷贝操作执行流程为例,具体实现步骤如下:Taking the copy operation execution process shown in Figure 5 as an example, the specific implementation steps are as follows:
201、VM(英文:Virual Machine,中文:虚拟机)1中的APP下发拷贝指令。201. The APP in VM (English: Virtual Machine, Chinese: virtual machine) 1 issues a copy command.
其中,拷贝指令具体为将文件A从目录a拷贝到VM3的目录b;VM1可以被视为源端虚拟机,VM3可以被视为目的虚拟机。Wherein, the copy instruction is specifically to copy file A from directory a to directory b of VM3; VM1 may be regarded as a source virtual machine, and VM3 may be regarded as a destination virtual machine.
需要说明的是,拷贝指令可以在同一虚拟机上实现,或是在不同虚拟机上实现。当拷贝指令在不同虚拟机上实现时,VM1与VM3为不同虚拟机;当拷贝指令在同一虚拟机上实现时,VM1与VM3为同一虚拟机。It should be noted that the copy instruction can be implemented on the same virtual machine or on different virtual machines. When the copy instruction is implemented on different virtual machines, VM1 and VM3 are different virtual machines; when the copy instruction is implemented on the same virtual machine, VM1 and VM3 are the same virtual machine.
202、VM1截获拷贝指令。202. VM1 intercepts the copy instruction.
203、VM1通知VM3创建目的空文件。203. VM1 notifies VM3 to create a destination empty file.
若VM1与VM3为同一虚拟机,那么当VM1(即VM3)截获拷贝指令之后,可以直接在本地的磁盘文件系统中创建目的空文件。需要说明的是,如图5所示的交互流程图为VM1与VM3属于不同虚拟机时的交互流程。If VM1 and VM3 are the same virtual machine, after VM1 (that is, VM3) intercepts the copy instruction, it can directly create a target empty file in the local disk file system. It should be noted that the interaction flowchart shown in FIG. 5 is an interaction flow when VM1 and VM3 belong to different virtual machines.
204、VM3在本地的磁盘文件系统中创建目的空文件。204. The VM3 creates a destination empty file in the local disk file system.
205、VM1初始化文件偏移f=0。205. VM1 initializes the file offset f=0.
考虑到文件可能存在多个数据块,而在一般情况下拷贝操作每次只能针对一个数据块进行,因此,在这里所设置的文件偏移f是用于在后续执行过程中判断是否继续进行其他数据块的拷贝操作的条件。也就意味着,直到文件A的偏移f与待拷贝的文件A的数据块的偏移相同时,结束拷贝操作。Considering that there may be multiple data blocks in the file, and in general, the copy operation can only be performed on one data block at a time, therefore, the file offset f set here is used to determine whether to continue in the subsequent execution process Conditions for copy operations of other data blocks. That is to say, until the offset f of the file A is the same as the offset of the data block of the file A to be copied, the copy operation ends.
需要说明的是,上述判别方式是按照文件中每个数据块的生成顺序依次进行拷贝操作的,但在本发明中不仅限于上述方式,也可以乱序进行拷贝操作,只要能够保证该文件中的所有数据块均完成拷贝即可。It should be noted that the above-mentioned discrimination method is to perform the copy operation sequentially according to the generation order of each data block in the file, but in the present invention, it is not limited to the above-mentioned method, and the copy operation can also be performed out of sequence, as long as the data blocks in the file can be guaranteed All data blocks are copied.
206、判断f是否大于或等于文件尾的偏移。206. Determine whether f is greater than or equal to the offset of the end of the file.
其中,当f大于或等于文件尾的偏移时,结束拷贝操作;否则,继续执行步骤207。Wherein, when f is greater than or equal to the offset of the end of the file, the copy operation ends; otherwise, continue to execute step 207 .
207、VM1中的文件系统获取偏移f对应数据块B(vlba,vlen)。207. The file system in VM1 acquires offset f corresponding to data block B (vlba, vlen).
其中,数据块B为待拷贝的文件A中的起始数据块,vlba为数据块B的起始虚拟源地址,vlen为数据块B的长度。Wherein, data block B is the initial data block in file A to be copied, vlba is the initial virtual source address of data block B, and vlen is the length of data block B.
208、VM3在文件系统中分配对应数据块C(vlba,vlen)。208. The VM3 allocates a corresponding data block C (vlba, vlen) in the file system.
为了保证数据块B能够完全复制到数据块C,在本发明中,数据块B和数据块C的vlen可以相同,或者,数据块C的长度大于数据块B的长度。需要说明的是,步骤208所示的分配操作只是在目的虚拟机,即在VM3的文件系统中分配,而不是真实下发到后端的真实存储设备中进行空间的分配。也就意味着,本发明所实现的操作方案可以不借助于实际物理存储设备。In order to ensure that data block B can be completely copied to data block C, in the present invention, the vlen of data block B and data block C can be the same, or the length of data block C is greater than the length of data block B. It should be noted that the allocation operation shown in step 208 is only allocated in the file system of the target virtual machine, that is, VM3, rather than actually distributed to the back-end real storage device for space allocation. That is to say, the operation scheme realized by the present invention may not rely on actual physical storage devices.
209、VM1生成拷贝卸载命令(拷贝,VM1.B.vlba,VM3.C.vlba,vlen)。209. VM1 generates a copy unload command (copy, VM1.B.vlba, VM3.C.vlba, vlen).
在本发明中,拷贝卸载命令的格式为操作类型,虚拟源地址,虚拟目的地址,长度。其中,操作类型为拷贝,虚拟源地址为VM1中数据块B的vlba,虚拟目的地址为VM3中数据块C的vlba,长度为数据块B的长度,同时也可以是数据块C的长度,即vlen。In the present invention, the format of the copy unload command is operation type, virtual source address, virtual destination address, and length. Wherein, the operation type is copy, the virtual source address is vlba of data block B in VM1, the virtual destination address is vlba of data block C in VM3, and the length is the length of data block B, and can also be the length of data block C at the same time, namely vlen.
210、VM1的前端进程将拷贝卸载命令经由IO环向主机的后端进程发送。210. The front-end process of VM1 sends the copy unload command to the back-end process of the host through the IO ring.
211、当主机的后端进程接收到拷贝卸载命令之后,解析拷贝卸载命令,得到解析后的拷贝卸载命令(拷贝,plba1,VM3.C.vlba,plen)。211. After receiving the copy uninstall command, the backend process of the host parses the copy uninstall command to obtain the parsed copy uninstall command (copy, plba1, VM3.C.vlba, plen).
其中,解析后的拷贝卸载命令的格式为操作类型,物理源地址,虚拟目的地址,长度。其中,操作类型为拷贝,物理源地址为数据块B的plba,即数据块B的物理地址,虚拟目的地址为VM3中数据块C的vlba,长度为数据块B在本地磁盘中对应的长度,即plen。Wherein, the format of the parsed copy unload command is an operation type, a physical source address, a virtual destination address, and a length. Wherein, the operation type is copy, the physical source address is the plba of data block B, that is, the physical address of data block B, the virtual destination address is the vlba of data block C in VM3, and the length is the corresponding length of data block B in the local disk, That is, plen.
212、主机将VM3.C.vlba指向plba1,并增加plba1对应数据块的引用计数。212. The host points VM3.C.vlba to plba1, and increases the reference count of the data block corresponding to plba1.
需要说明的是,步骤212仅提供了一种具体的对待操作数据执行指定操作的方式,即当上述实现方式用于分布式共享存储系统时,VM1、VM3与主机是如何实现上述过程的,而对于当本发明所提供的方法应用于集群文件系统时,可参见步骤1053至步骤1055,在此不做赘述。另外,在步骤212中,VM3.C.vlba指向plba1可以视为建立了VM3.C.vlba和plba1之间的映射关系。It should be noted that step 212 only provides a specific way to perform a specified operation on the data to be operated, that is, when the above implementation is used in a distributed shared storage system, how VM1, VM3 and the host implement the above process, and For when the method provided by the present invention is applied to the cluster file system, refer to step 1053 to step 1055, which will not be repeated here. In addition, in step 212, VM3.C.vlba pointing to plba1 can be regarded as establishing a mapping relationship between VM3.C.vlba and plba1.
213、主机将拷贝结果返回VM1。213. The host returns the copy result to VM1.
214、在VM1中,f+=B.vlen。214. In VM1, f+=B.vlen.
需要说明的是,B.vlen为数据块B的长度;仅当待拷贝的文件A完全拷贝到指定位置,即VM3的文件系统中之后,才能结束上述操作流程,否则将循环执行上述步骤206至步骤214,直到遍历到文件A的文件尾,也就是直至将文件中所有数据块拷贝完。It should be noted that B.vlen is the length of data block B; the above operation process can only be ended when the file A to be copied is completely copied to the specified location, that is, in the file system of VM3, otherwise the above steps 206 to Step 214, until the end of file A is traversed, that is, until all data blocks in the file are copied.
考虑到在本发明所提供的技术方案应用于不同数据中心结构时,会有不同的拷贝方法,在本发明实施例的一个实现方式中,当本发明的方法应用于分布式共享存储系统时,可以通过改变地址之间的映射关系,并增加待操作数据的物理源地址对应的数据块的引用计数,来完成拷贝操作。因此,在上述实现方式的基础上,步骤105主机根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作,可以具体实现为步骤1051和步骤1052:Considering that when the technical solution provided by the present invention is applied to different data center structures, there will be different copy methods, in an implementation of the embodiment of the present invention, when the method of the present invention is applied to a distributed shared storage system, The copy operation can be completed by changing the mapping relationship between addresses and increasing the reference count of the data block corresponding to the physical source address of the data to be operated. Therefore, on the basis of the above implementation, in step 105, the host performs specified operations on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated, which can be specifically implemented as steps 1051 and 1052:
1051、主机将待操作数据的虚拟目的地址更新为待操作数据的物理源地址。1051. The host updates the virtual destination address of the data to be operated to the physical source address of the data to be operated.
1052、主机根据更新的虚拟目的地址,将待操作数据的物理源地址对应的数据块的引用计数的数值加一。1052. The host adds one to the value of the reference count of the data block corresponding to the physical source address of the data to be operated according to the updated virtual destination address.
其中,引用计数的值用于表示待操作数据的物理地址对应的数据块被引用的次数。Wherein, the value of the reference count is used to indicate the number of times the data block corresponding to the physical address of the data to be operated is referenced.
以如图5所示的拷贝操作执行流程为例,当方法应用于分布式共享存储系统时,主机可以建立VM3.C.vlba与plba1之间的映射关系,并增加plba1对应数据块的引用计数,从而完成数据块B的拷贝操作。Take the copy operation execution process shown in Figure 5 as an example, when the method is applied to a distributed shared storage system, the host can establish a mapping relationship between VM3.C.vlba and plba1, and increase the reference count of the data block corresponding to plba1 , thus completing the copy operation of data block B.
在本发明中,采用上述方式来实现待操作数据的拷贝操作,仅仅建立了地址之间的映射关系,并增加待操作数据的物理源地址的引用计数。因此,对于虚拟机而言,降低了虚拟机存储操作对虚拟机计算网络资源的消耗,同时,通过增加引用计数还起到了加速的效果。In the present invention, the copy operation of the data to be operated is realized by adopting the above method, only the mapping relationship between addresses is established, and the reference count of the physical source address of the data to be operated is increased. Therefore, for the virtual machine, the consumption of the computing network resources of the virtual machine by the storage operation of the virtual machine is reduced, and at the same time, an acceleration effect is also achieved by increasing the reference count.
另外,当本发明的方法应用于集群文件系统时,可以通过将待操作数据的物理源地址对应的数据块进行拷贝,之后将映射关系中的物理源地址改变为物理目的地址,来完成拷贝操作。因此,在上述实现方式的基础上,步骤105主机根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作,还可以具体实现为步骤1053至步骤1055:In addition, when the method of the present invention is applied to a cluster file system, the copy operation can be completed by copying the data block corresponding to the physical source address of the data to be operated, and then changing the physical source address in the mapping relationship to the physical destination address . Therefore, on the basis of the above implementation, in step 105, the host performs specified operations on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated, which can also be specifically implemented as steps 1053 to 1055:
1053、主机建立待操作数据的虚拟目的地址与待操作数据的物理源地址之间的映射关系。1053. The host establishes a mapping relationship between the virtual destination address of the data to be operated and the physical source address of the data to be operated.
1054、主机将待操作数据的物理源地址对应的数据块拷贝到待操作数据的物理目的地址。1054. The host copies the data block corresponding to the physical source address of the data to be operated to the physical destination address of the data to be operated.
1055、主机将映射关系中的待操作数据的物理源地址替换为待操作数据的物理目的地址。1055. The host replaces the physical source address of the data to be operated in the mapping relationship with the physical destination address of the data to be operated.
以如图5所示的拷贝操作执行流程为例,当方法应用于集群文件系统时,主机可以将plba1的数据块拷贝到plba2,并建立VM3.C.vlba与plba2之间的映射关系,从而完成数据块B的拷贝操作。Take the copy operation execution process shown in Figure 5 as an example, when the method is applied to the cluster file system, the host can copy the data block of plba1 to plba2, and establish the mapping relationship between VM3.C.vlba and plba2, so that Complete the copy operation of data block B.
在本发明中,采用上述方式来实现待操作数据的拷贝操作,需要先将数据块从物理源地址拷贝到物理目的地址,之后建立虚拟源地址与物理目的地址之间的映射关系。对于虚拟机而言,降低了虚拟机存储操作对虚拟机计算网络资源的消耗,但相比较于步骤1051和步骤1052所示的实现方案,无法达到有效的拷贝加速的效果。In the present invention, the above method is used to realize the copy operation of the data to be operated. It is necessary to first copy the data block from the physical source address to the physical destination address, and then establish the mapping relationship between the virtual source address and the physical destination address. For the virtual machine, the consumption of virtual machine computing network resources by the storage operation of the virtual machine is reduced, but compared with the implementation solutions shown in steps 1051 and 1052, effective copy acceleration cannot be achieved.
由于在数据结构中心中,待操作数据的存储方式可能为块存储或是对象存储,而在进行拷贝操作之前,需要针对不同存储方式来完成待操作数据的虚拟源地址的转换操作,在本发明实施例的一个实现方式中,分别针对不同存储方式提供了具体的将待操作数据的虚拟源地址进行转换的方案。因此,在上述实现方式的基础上,步骤104主机将卸载命令中的虚拟源地址转换为待操作数据的物理源地址,可以具体实现为步骤1041或步骤1042:Since in the data structure center, the storage method of the data to be operated may be block storage or object storage, and before the copy operation, it is necessary to complete the conversion operation of the virtual source address of the data to be operated according to different storage methods, in the present invention In an implementation of the embodiment, specific solutions for converting the virtual source address of the data to be operated are provided for different storage methods. Therefore, on the basis of the above implementation, the host in step 104 converts the virtual source address in the unload command into the physical source address of the data to be operated, which can be specifically implemented as step 1041 or step 1042:
1041、在待操作数据的存储方式为块存储的情况下,主机将待操作数据的虚拟源地址转换为待操作数据的物理源地址。1041. When the storage mode of the data to be operated is block storage, the host converts the virtual source address of the data to be operated into the physical source address of the data to be operated.
1042、在待操作数据的存储方式为对象存储的情况下,主机将待操作数据的虚拟源地址转换为对象标识。1042. When the storage mode of the data to be operated is object storage, the host converts the virtual source address of the data to be operated into an object identifier.
其中,对象标识为待操作数据对应的数据块的标识。Wherein, the object identifier is an identifier of a data block corresponding to the data to be operated.
需要说明的是,在主机的后端进程解析卸载命令时,需要将vlba、vlen进行转换,后端进程具体可以将存储方式为块存储的待操作数据的虚拟源地址vlba,转换为物理源地址plba,或是将存储方式为对象存储的待操作数据的虚拟源地址vlba,转换为对象标识oid。It should be noted that when the backend process of the host parses the unload command, vlba and vlen need to be converted, and the backend process can specifically convert the virtual source address vlba of the data to be operated in block storage into a physical source address plba, or convert the virtual source address vlba of the data to be operated in object storage into an object identifier oid.
在本发明中,在主机的后端进程进行卸载命令解析的过程中,会将待操作数据的虚拟源地址进行转换,之后交给分布式共享存储系统或是集群文件系统进行处理。In the present invention, in the process of unloading command parsing by the backend process of the host, the virtual source address of the data to be operated is converted, and then handed over to the distributed shared storage system or the cluster file system for processing.
当虚拟机中的应用程序下发的操作指令所指示的操作为清零时,在本发明实施例的一个实现方式中,可以根据虚拟磁盘的模拟情况来选择合适的清零方法。因此,在如图4所示的实现方式的基础上,步骤105主机根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作,还可以具体实现为步骤1056至步骤1059中的一项:When the operation instruction issued by the application program in the virtual machine indicates that the operation is clearing, in an implementation manner of the embodiment of the present invention, an appropriate zeroing method may be selected according to the simulation situation of the virtual disk. Therefore, on the basis of the implementation shown in FIG. 4 , in step 105, the host performs specified operations on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated. One item in step 1059:
1056、当虚拟磁盘为待操作数据模拟的虚拟磁盘时,主机将待操作数据的物理源地址的比特图中每一位清零。1056. When the virtual disk is simulated by the data to be operated, the host clears each bit in the bitmap of the physical source address of the data to be operated to zero.
1057、当虚拟磁盘为分布式共享存储系统模拟的虚拟磁盘时,主机将待操作数据的对象标识释放,并释放待操作数据对应的数据块。1057. When the virtual disk is simulated by the distributed shared storage system, the host releases the object identifier of the data to be operated, and releases the data block corresponding to the data to be operated.
1058、当虚拟磁盘为集群文件系统模拟的虚拟磁盘时,主机将待操作数据的数据块释放。1058. When the virtual disk is a virtual disk simulated by the cluster file system, the host releases the data block of the data to be operated.
1059、当不存在待操作数据的虚拟源地址与待操作数据的物理源地址之间的映射关系时,主机将待操作数据的物理源地址对应的数据块写零。1059. When there is no mapping relationship between the virtual source address of the data to be operated and the physical source address of the data to be operated, the host writes zero to the data block corresponding to the physical source address of the data to be operated.
例如:如果虚拟磁盘是由vhd(英文:Microsoft Virtual Hard Disk format,中文:虚拟磁盘格式)文件进行模拟的,则将vhd文件中对应plba地址的bitmap清0;如果虚拟磁盘是由Server San提供的,则将Server San中对应块的对象标识释放,并释放对应数据块;如果虚拟磁盘是由集群文件系统提供的,则将集群文件系统中plba对应文件的数据块释放;如果不存在元数据,则将plba对应的数据块写0,而这种方法只能降低虚拟机存储操作对虚拟机计算网络资源的消耗,却无法像上述三种实现方法那样起到加速的效果。For example: if the virtual disk is simulated by a vhd (English: Microsoft Virtual Hard Disk format, Chinese: virtual disk format) file, clear the bitmap corresponding to the plba address in the vhd file to 0; if the virtual disk is provided by Server San , then release the object identifier of the corresponding block in Server San, and release the corresponding data block; if the virtual disk is provided by the cluster file system, release the data block of the file corresponding to plba in the cluster file system; if there is no metadata, Then write 0 to the data block corresponding to plba, and this method can only reduce the consumption of virtual machine computing network resources by virtual machine storage operations, but cannot achieve the acceleration effect like the above three implementation methods.
以如图6所示的清零操作执行流程为例,具体实现步骤如下:Taking the execution process of the clearing operation as shown in Figure 6 as an example, the specific implementation steps are as follows:
301、VM1中的APP下发清零指令。301. The APP in VM1 issues a clearing instruction.
其中,清零指令具体可以为对文件A中f偏移下发写0操作。Wherein, the clearing command may specifically be an operation of writing 0 to the f offset in file A.
302、VM1截获清零指令,并将清零指令转换为清零卸载命令(清零,vlba,vlen)。302. VM1 intercepts the clear instruction, and converts the clear instruction into a clear unload command (clear, vlba, vlen).
在本发明中,清零卸载命令的格式为操作类型,虚拟源地址,长度。其中,操作类型为清零,虚拟源地址为VM1中数据块B的vlba,长度为数据块B的长度,即vlen。In the present invention, the format of the clear unload command is operation type, virtual source address, and length. Wherein, the operation type is clear, the virtual source address is vlba of data block B in VM1, and the length is the length of data block B, namely vlen.
303、VM1的前端进程将清零卸载命令经由IO环向后端进程发送。303. The front-end process of VM1 sends a clearing and unloading command to the back-end process via the IO ring.
304、当主机的后端进程接收到清零卸载命令之后,解析清零卸载命令,得到解析后的清零卸载命令(清零,plba1,plen)。304. After receiving the clearing and uninstalling command, the backend process of the host parses the clearing and uninstalling command, and obtains the parsed clearing and uninstalling command (clearing, plba1, plen).
其中,解析后的清零卸载命令的格式为操作类型,物理源地址,长度。其中,操作类型为清零,物理源地址为数据块B的plba,长度为数据块B在本地磁盘中对应的长度,即plen。Wherein, the format of the parsed clearing and unloading command is an operation type, a physical source address, and a length. Wherein, the operation type is clear, the physical source address is the plba of the data block B, and the length is the corresponding length of the data block B in the local disk, that is, plen.
305、主机对plba,plen所对应的元数据执行清零操作。305. The host performs a clearing operation on metadata corresponding to plba and plen.
其中,元数据用于表示虚拟源地址与物理源地址之间的映射关系。需要说明的是,在本发明中,具体执行清零操作的过程包括但不仅限于上述步骤1056至步骤1059中所示的针对不同情况所采取的不同清零方法,在此不做赘述。Wherein, the metadata is used to represent the mapping relationship between the virtual source address and the physical source address. It should be noted that, in the present invention, the specific clearing operation process includes, but is not limited to, different clearing methods for different situations shown in steps 1056 to 1059 above, which will not be repeated here.
306、主机将清零结果返回VM1。306. The host returns the zeroing result to VM1.
在本发明中,在虚拟机获取到应用程序下发的操作指令为指示清零操作的指令时,主机可以根据虚拟磁盘的模拟情况来选择合适的清零方法,之后对待操作数据执行清零操作。In the present invention, when the virtual machine acquires that the operation instruction issued by the application program is an instruction indicating a clearing operation, the host can select an appropriate clearing method according to the simulation situation of the virtual disk, and then perform a clearing operation on the data to be operated .
上述主要从虚拟机和主机交互的角度对本发明实施例提供的方案进行了介绍。可以理解的是,为了实现上述功能,本发明还提供了一种命令卸载装置80,如图7所示,该装置80包含了执行各个功能相应的软件模块。本领域技术人员应该很容易意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,本发明能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。The foregoing mainly introduces the solution provided by the embodiment of the present invention from the perspective of interaction between the virtual machine and the host. It can be understood that, in order to realize the above functions, the present invention also provides a command unloading device 80, as shown in FIG. 7, the device 80 includes corresponding software modules for performing various functions. Those skilled in the art should easily realize that the present invention can be realized in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.
本发明实施例可以根据上述方法示例对装置80进行功能模块的划分,例如,可以对应各个功能划分各个功能模块,也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现,也可以采用软件功能模块的形式实现。需要说明的是,本发明实施例中对模块的划分是示意性的,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式。In the embodiment of the present invention, the device 80 may be divided into functional modules according to the above method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present invention is schematic, and is only a logical function division, and there may be another division manner in actual implementation.
在采用对应各个功能划分各个功能模块的情况下,图7示出了上述实施例中所涉及的装置80的一种可能的结构示意图,装置80包括:虚拟机81和主机82。虚拟机81,用于截获应用程序下发的操作指令,操作指令用于指示对待操作数据执行指定操作;虚拟机81,还用于根据操作指令,生成卸载命令,并向主机82发送卸载命令,卸载命令包括拷贝卸载命令或清零卸载命令;主机82,用于将卸载命令中的虚拟源地址转换为待操作数据的物理源地址,虚拟源地址用于表示待操作数据在虚拟机81的虚拟磁盘中的位置;主机82,还用于根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作。In the case of dividing each functional module corresponding to each function, FIG. 7 shows a possible structural diagram of an apparatus 80 involved in the above embodiment. The apparatus 80 includes: a virtual machine 81 and a host 82 . The virtual machine 81 is used to intercept the operation instruction issued by the application program, and the operation instruction is used to instruct to perform a specified operation on the data to be operated; the virtual machine 81 is also used to generate an uninstall command according to the operation instruction, and send the uninstall command to the host 82, The unloading command includes a copy unloading command or a clear unloading command; the host 82 is used to convert the virtual source address in the unloading command into the physical source address of the data to be operated, and the virtual source address is used to represent the virtual source address of the data to be operated in the virtual machine 81. The location in the disk; the host 82 is also used to perform specified operations on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated.
在本发明中,虚拟机81不仅用于执行图4中的过程101至过程103,还用于执行图5和图6中VM1所执行的所有过程;主机82不仅用于执行图4中的过程104和过程105,还用于执行图5和图6中主机所执行的所有过程。其中,上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述,在此不再赘述。此外,装置80中的主机82和虚拟机81,还可以用于执行本文所描述的技术的其它过程。在本发明中,装置80中还可以包括用于与外界其他设备进行通信的通信模块83,以及用于存储程序代码和数据存储模块84。其中,该通信模块83具体可以是收发器、收发电路或通信接口等,该存储模块84具体可以是存储器,主机82和虚拟机81可以集成在该存储模块84中,或者,该存储模块84可以为至少两个,并分别设置在主机82和虚拟机81内部。此外,在装置80中,还可以包括处理模块85。其中,该处理模块85可以是处理器或控制器,例如可以是中央处理器,通用处理器,数字信号处理器(英文:Digital Signal Processor,简称:DSP),专用集成电路(英文:Application-Specific Integrated Circuit,简称:ASIC),现场可编程门阵列(英文:FieldProgrammable Gate Array,简称:FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本发明公开内容所描述的各种示例性的逻辑方框,模块和电路。所述处理器也可以是实现计算功能的组合,例如包含一个或多个微处理器组合,DSP和微处理器的组合等等。In the present invention, the virtual machine 81 is not only used to execute the process 101 to the process 103 in FIG. 4, but also used to execute all the processes performed by VM1 in FIG. 5 and FIG. 6; the host computer 82 is not only used to execute the process in FIG. 4 104 and process 105 are also used to execute all processes executed by the host in FIG. 5 and FIG. 6 . Wherein, all relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding function module, and will not be repeated here. In addition, the host 82 and the virtual machine 81 in the device 80 can also be used to execute other processes of the techniques described herein. In the present invention, the device 80 may further include a communication module 83 for communicating with other external devices, and a storage module 84 for storing program codes and data. Wherein, the communication module 83 may specifically be a transceiver, a transceiver circuit, or a communication interface, etc., the storage module 84 may specifically be a memory, and the host 82 and the virtual machine 81 may be integrated in the storage module 84, or the storage module 84 may be There are at least two, and they are set inside the host machine 82 and the virtual machine 81 respectively. In addition, the device 80 may further include a processing module 85 . Wherein, the processing module 85 can be a processor or a controller, such as a central processing unit, a general-purpose processor, a digital signal processor (English: Digital Signal Processor, referred to as: DSP), an application-specific integrated circuit (English: Application-Specific Integrated Circuit, referred to as: ASIC), field programmable gate array (English: Field Programmable Gate Array, referred to as: FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, for example, a combination of one or more microprocessors, a combination of DSP and a microprocessor, and so on.
本发明提供的命令卸载装置,相比较于现有技术中借助物理存储设备来执行指定操作,在本发明中,虚拟机可以根据应用程序所下发的操作指令来生成卸载命令,之后由虚拟机将卸载命令向主机发送;主机在接收到虚拟机发送的卸载命令之后,将卸载命令所携带的虚拟源地址转换为待操作数据的物理源地址,之后,主机可以根据卸载命令所携带的虚拟目的地址和上述待操作数据的物理源地址,对待操作数据执行指定操作。这样虽然需要虚拟机与主机共同来完成指定操作的执行过程,但由于主机是根据虚拟机生成的卸载命令来完成指定操作的,因此,无论当前应用本发明的系统为集群文件系统、分布式共享存储系统,或是其他系统,主机均能够识别该卸载命令。并且,主机和虚拟机均属于物理机的一部分,在上述执行过程中,主要由主机在接收到卸载命令之后执行相应操作,而无需物理存储设备执行相应的操作,因此,采用本发明所提供的实现方式,能够在处理操作指令(如拷贝或者清零指令)时,有效减少对物理机的硬件资源的浪费。Compared with the command unloading device provided by the present invention, which uses a physical storage device to perform specified operations in the prior art, in the present invention, the virtual machine can generate an unloading command according to the operation instructions issued by the application program, and then the virtual machine Send the unloading command to the host; after the host receives the unloading command sent by the virtual machine, it converts the virtual source address carried by the unloading command into the physical source address of the data to be operated, and then the host can The address and the physical source address of the data to be operated are used to perform the specified operation on the data to be operated. Although the execution process of the specified operation needs to be completed by the virtual machine and the host computer together like this, because the host computer completes the specified operation according to the unloading command generated by the virtual machine, no matter whether the system currently applying the present invention is a cluster file system, distributed shared The storage system, or other systems, and the host can recognize the uninstall command. Moreover, both the host and the virtual machine are part of the physical machine. In the above execution process, the host performs corresponding operations after receiving the unloading command, without the need for a physical storage device to perform corresponding operations. Therefore, using the The implementation manner can effectively reduce the waste of hardware resources of the physical machine when processing operation instructions (such as copy or clear instructions).
当通信模块83为输入设备和输出设备,虚拟机81和主机82集成在同一存储模块84上,则该存储模块可以具体为处理器,存储模块84为存储器时,本发明实施例所涉及的物理机可以为图8所示的物理机90。When the communication module 83 is an input device and an output device, and the virtual machine 81 and the host 82 are integrated on the same storage module 84, the storage module can be specifically a processor, and when the storage module 84 is a memory, the physical The machine may be the physical machine 90 shown in FIG. 8 .
参阅图8所示,该物理机90包括:处理器91、输入设备92、输出设备93、存储器94以及总线95。其中,处理器91、输入设备92、输出设备93,以及存储器94通过总线95相互连接,存储器94中具体可以包括主机941和虚拟机942;总线95可以是外设部件互连标准(英文:Peripheral Component Interconnect,简称:PCI)总线或扩展工业标准结构(英文:Extended Industry Standard Architecture,简称:EISA)总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示,图8中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。Referring to FIG. 8 , the physical machine 90 includes: a processor 91 , an input device 92 , an output device 93 , a memory 94 and a bus 95 . Wherein, the processor 91, the input device 92, the output device 93, and the memory 94 are connected to each other through a bus 95, and the memory 94 may specifically include a host computer 941 and a virtual machine 942; the bus 95 may be a peripheral component interconnection standard (English: Peripheral Component Interconnect (PCI for short) bus or Extended Industry Standard Architecture (English: Extended Industry Standard Architecture, EISA for short) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 8 , but it does not mean that there is only one bus or one type of bus.
结合本发明公开内容所描述的方法或者算法的步骤可以硬件的方式来实现,也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成,软件模块可以被存放于随机存取存储器(英文:Random Access Memory,简称:RAM)、闪存、只读存储器(英文:Read Only Memory,简称:ROM)、可擦除可编程只读存储器(英文:ErasableProgrammable ROM,简称:EPROM)、电可擦可编程只读存储器(英文:Electrically EPROM,简称:EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(简称:CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器,从而使处理器能够从该存储介质读取信息,且可向该存储介质写入信息。当然,存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外,该ASIC可以位于核心网接口设备中。当然,处理器和存储介质也可以作为分立组件存在于核心网接口设备中。The steps of the methods or algorithms described in conjunction with the disclosure of the present invention may be implemented in the form of hardware, or may be implemented in the form of a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (English: Random Access Memory, abbreviated as RAM), flash memory, read-only memory (English: Read Only Memory, abbreviated: ROM), and Erasable Programmable Read-Only Memory (English: Erasable Programmable ROM, referred to as: EPROM), Electrically Erasable Programmable Read-Only Memory (English: Electrically EPROM, referred to as: EEPROM), registers, hard disk, mobile hard disk, CD-ROM (abbreviated: CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be a component of the processor. The processor and storage medium can be located in the ASIC. In addition, the ASIC may be located in the core network interface device. Certainly, the processor and the storage medium may also exist in the core network interface device as discrete components.
需要说明的是,上述输入设备92、输出设备93可以根据物理机90是否需要与其他设备进行通信来确定是否需要设置。图8所示的主机941作为管理层,用以完成硬件资源的管理、分配;为虚拟机942呈现虚拟硬件平台;实现虚拟机的调度和隔离。其中,主机941可能是虚拟机监控器(简称:VMM);此外,有时VMM和一个特权虚拟机配合,两者结合组成主机941。其中,虚拟硬件平台对其上运行的各个虚拟机提供各种硬件资源,如提供虚拟处理器、内存、虚拟磁盘、虚拟网卡等等。其中,该虚拟磁盘可对应主机941的一个文件或者一个逻辑块设备。虚拟机942则运行在主机941为其准备的虚拟硬件平台上,主机941上运行一个或多个诸如942的虚拟机。其中,虚拟机942则通过虚拟机软件可以在一台物理机上模拟出一台或者多台虚拟的计算机,而这些虚拟机就像真正的计算机那样进行工作,虚拟机942上可以安装操作系统和应用程序,虚拟机942还可访问网络资源。对于在虚拟机942中运行的应用程序而言,虚拟机就像是在真正的计算机中进行工作。It should be noted that, whether the input device 92 and the output device 93 need to be set can be determined according to whether the physical machine 90 needs to communicate with other devices. The host 941 shown in FIG. 8 acts as a management layer to complete the management and allocation of hardware resources; present a virtual hardware platform for the virtual machine 942; and realize the scheduling and isolation of virtual machines. Wherein, the host 941 may be a virtual machine monitor (VMM for short); in addition, sometimes the VMM cooperates with a privileged virtual machine, and the combination of the two forms the host 941 . Wherein, the virtual hardware platform provides various hardware resources for each virtual machine running on it, such as virtual processor, memory, virtual disk, virtual network card and so on. Wherein, the virtual disk may correspond to a file or a logical block device of the host 941 . The virtual machine 942 runs on the virtual hardware platform prepared for it by the host 941, and one or more virtual machines such as 942 run on the host 941. Among them, the virtual machine 942 can simulate one or more virtual computers on a physical machine through the virtual machine software, and these virtual machines work like real computers, and the operating system and applications can be installed on the virtual machine 942. program, the virtual machine 942 can also access network resources. For applications running in the virtual machine 942, the virtual machine is like working on a real computer.
在本发明中,虚拟机942,用于截获应用程序下发的操作指令,操作指令用于指示对待操作数据执行指定操作;虚拟机942,还用于根据操作指令,生成卸载命令,并向主机941发送卸载命令,卸载命令包括拷贝卸载命令或清零卸载命令;主机941,用于将卸载命令中的虚拟源地址转换为待操作数据的物理源地址,虚拟源地址用于表示待操作数据在虚拟机942的虚拟磁盘中的位置;主机941,还用于根据卸载命令中的虚拟目的地址和待操作数据的物理源地址,对待操作数据执行指定操作。In the present invention, the virtual machine 942 is used to intercept the operation instruction issued by the application program, and the operation instruction is used to instruct to perform a specified operation on the data to be operated; the virtual machine 942 is also used to generate an uninstall command according to the operation instruction, and send the command to the host 941 sends an unload command, and the unload command includes a copy unload command or a clear unload command; the host 941 is used to convert the virtual source address in the unload command into the physical source address of the data to be operated, and the virtual source address is used to indicate that the data to be operated is in The location in the virtual disk of the virtual machine 942; the host 941 is also used to perform a specified operation on the data to be operated according to the virtual destination address in the unload command and the physical source address of the data to be operated.
此外,虚拟机942还可以用于执行如图5和图6中虚拟机所实现的功能,主机941还可以用于执行如图5和图6中主机所实现的功能,具体可以参见上述方法实施例中的各个步骤,在此不做赘述。In addition, the virtual machine 942 can also be used to perform the functions implemented by the virtual machine in Figure 5 and Figure 6, and the host 941 can also be used to perform the functions implemented by the host in Figure 5 and Figure 6, for details, please refer to the above method implementation Each step in the example will not be repeated here.
本发明提供的物理机,相比较于现有技术中借助物理存储设备来执行指定操作,在本发明中,虚拟机可以根据应用程序所下发的操作指令来生成卸载命令,之后由虚拟机将卸载命令向主机发送;主机在接收到虚拟机发送的卸载命令之后,将卸载命令所携带的虚拟源地址转换为待操作数据的物理源地址,之后,主机可以根据卸载命令所携带的虚拟目的地址和上述待操作数据的物理源地址,对待操作数据执行指定操作。这样虽然需要虚拟机与主机共同来完成指定操作的执行过程,但由于主机是根据虚拟机生成的卸载命令来完成指定操作的,因此,无论当前应用本发明的系统为集群文件系统、分布式共享存储系统,或是其他系统,主机均能够识别该卸载命令。并且,主机和虚拟机均属于物理机的一部分,在上述执行过程中,主要由主机在接收到卸载命令之后执行相应操作,而无需物理存储设备执行相应的操作,因此,采用本发明所提供的实现方式,能够在处理操作指令(如拷贝或者清零指令)时,有效减少对物理机的硬件资源的浪费。Compared with the physical machine provided by the present invention, which uses a physical storage device to perform specified operations in the prior art, in the present invention, the virtual machine can generate an uninstall command according to the operation instructions issued by the application program, and then the virtual machine will The unloading command is sent to the host; after the host receives the unloading command sent by the virtual machine, it converts the virtual source address carried by the unloading command into the physical source address of the data to be operated, and then the host can use the virtual destination address carried by the unloading command and the physical source address of the above data to be operated, perform the specified operation on the data to be operated. Although the execution process of the specified operation needs to be completed by the virtual machine and the host computer together like this, because the host computer completes the specified operation according to the unloading command generated by the virtual machine, no matter whether the system currently applying the present invention is a cluster file system, distributed shared The storage system, or other systems, and the host can recognize the uninstall command. Moreover, both the host and the virtual machine are part of the physical machine. In the above execution process, the host performs corresponding operations after receiving the unloading command, without the need for a physical storage device to perform corresponding operations. Therefore, using the The implementation manner can effectively reduce the waste of hardware resources of the physical machine when processing operation instructions (such as copy or clear instructions).
本领域技术人员应该可以意识到,在上述一个或多个示例中,本发明所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时,可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机存储介质和通信介质,其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。Those skilled in the art should be aware that, in the above one or more examples, the functions described in the present invention may be implemented by hardware, software, firmware or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
以上所述的具体实施方式,对本发明的目的、技术方案和有益效果进行了进一步详细说明,所应理解的是,以上所述仅为本发明的具体实施方式而已,并不用于限定本发明的保护范围,凡在本发明的技术方案的基础之上,所做的任何修改、等同替换、改进等,均应包括在本发明的保护范围之内。The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Protection scope, any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the protection scope of the present invention.
Claims (21)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610853875.5A CN106502721B (en) | 2016-09-26 | 2016-09-26 | A command unloading method, device and physical machine |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610853875.5A CN106502721B (en) | 2016-09-26 | 2016-09-26 | A command unloading method, device and physical machine |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106502721A CN106502721A (en) | 2017-03-15 |
CN106502721B true CN106502721B (en) | 2019-11-15 |
Family
ID=58290559
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610853875.5A Active CN106502721B (en) | 2016-09-26 | 2016-09-26 | A command unloading method, device and physical machine |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106502721B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107729049A (en) * | 2017-09-13 | 2018-02-23 | 深信服科技股份有限公司 | Instruct conversion method, equipment and computer-readable recording medium |
CN112015350B (en) * | 2020-08-28 | 2022-03-22 | 北京浪潮数据技术有限公司 | Method, device and medium for unloading data disk of virtual machine |
CN119917114A (en) * | 2023-10-31 | 2025-05-02 | 华为技术有限公司 | Method, device and related products for uninstalling microcode program |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7299266B2 (en) * | 2002-09-05 | 2007-11-20 | International Business Machines Corporation | Memory management offload for RDMA enabled network adapters |
US8250254B2 (en) * | 2007-07-31 | 2012-08-21 | Intel Corporation | Offloading input/output (I/O) virtualization operations to a processor |
CN101751274B (en) * | 2008-12-18 | 2012-10-10 | 联想(北京)有限公司 | Virtual machine system and unloading method of universal serial bus device thereof |
US8386745B2 (en) * | 2009-07-24 | 2013-02-26 | Advanced Micro Devices, Inc. | I/O memory management unit including multilevel address translation for I/O and computation offload |
US9389895B2 (en) * | 2009-12-17 | 2016-07-12 | Microsoft Technology Licensing, Llc | Virtual storage target offload techniques |
-
2016
- 2016-09-26 CN CN201610853875.5A patent/CN106502721B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN106502721A (en) | 2017-03-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN113287286B (en) | Input/output processing in distributed storage nodes over RDMA | |
US10620932B2 (en) | Replacing an accelerator firmware image without operating system reboot | |
US7484029B2 (en) | Method, apparatus, and computer usable program code for migrating virtual adapters from source physical adapters to destination physical adapters | |
CN111679889B (en) | Conversion migration method and system of virtual machine | |
US10228993B2 (en) | Data dump for a memory in a data processing system | |
CN116886719B (en) | Data processing method and device of storage system, equipment and medium | |
US10831684B1 (en) | Kernal driver extension system and method | |
CN112352221A (en) | Shared memory mechanism to support fast transfer of SQ/CQ pair communications between SSD device drivers and physical SSDs in virtualized environments | |
EP4170492A1 (en) | System and interrupt processing method | |
CN114691300A (en) | Hot migration method of virtual machine instance | |
CN105739930A (en) | Storage framework as well as initialization method, data storage method and data storage and management apparatus therefor | |
CN106815067B (en) | Online migration method and device for virtual machine with I/O virtualization | |
CN106502721B (en) | A command unloading method, device and physical machine | |
KR102326280B1 (en) | Method, apparatus, device and medium for processing data | |
KR102315102B1 (en) | Method, device, apparatus, and medium for booting a virtual machine | |
CN104123173B (en) | A kind of method and device for realizing inter-virtual machine communication | |
US20230236889A1 (en) | Distributed accelerator | |
CN111767082A (en) | Computing chip startup method, device and computer system | |
JP6954535B2 (en) | Communication device | |
CN104572315A (en) | Inter-subsystem communication method, communication entities and distributed communication system | |
CN113703913B (en) | Equipment testing method and device | |
CN115617270A (en) | Data processing method and device for virtualized storage equipment of data processor | |
JP2010231295A (en) | Analysis system | |
US10241821B2 (en) | Interrupt generated random number generator states | |
KR102865023B1 (en) | System and interrupt handling methods |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |