CN111338751B - Cross-pool migration method and device for data in same ceph cluster - Google Patents
Cross-pool migration method and device for data in same ceph cluster Download PDFInfo
- Publication number
- CN111338751B CN111338751B CN202010090739.1A CN202010090739A CN111338751B CN 111338751 B CN111338751 B CN 111338751B CN 202010090739 A CN202010090739 A CN 202010090739A CN 111338751 B CN111338751 B CN 111338751B
- Authority
- CN
- China
- Prior art keywords
- volume
- data
- destination
- instance
- nova
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a device for data cross-pool migration in the same ceph cluster. When the circle is used for judging that the state of the first volume of the first pool in the ceph cluster is mounted, transmitting the data migration related parameters to nova; and verifying the parameter by using nova to verify whether the data in the first volume can be migrated to the destination volume, if so, updating the volume information of the virtual machine to enable the state of the first volume to be in unloading and the state of the destination volume of the destination pool in the cluster to be mounted on the first instance, calling QEMU for data migration, merging the data which needs to be written into the destination volume during migration into the QEMU, updating the mounting information of the first instance and unloading the first volume from the QEMU. The scheme can perform data cross-pool migration on the mounted volumes.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for data cross-pool migration in the same ceph cluster.
Background
With the development of information technology and cloud computing, cloud computing technology is increasingly used in many fields of education, science, culture, government, and the like. The volume serves as a carrier of data, and the data in the volume needs to be backed up, migrated or restored in many cases.
Currently, in an OpenStack environment, a shader component of OpenStack can be used to directly perform data cross-pool migration on an uninstalled volume in the same ceph cluster.
In addition to the volume that is not mounted, in order to realize flexible operation on data, data cross-pool migration is generally performed on the mounted volume.
Disclosure of Invention
The invention provides a data cross-pool migration method and device in the same ceph cluster, which can perform data cross-pool migration on mounted volumes.
In order to achieve the purpose, the invention is realized by the following technical scheme:
in a first aspect, the invention provides a method for data cross-pool migration in a same ceph cluster, comprising:
utilizing a shader component of an OpenStack cloud computing management platform, judging whether the state of a first volume included by a first pool in a ceph cluster of data to be migrated is mounted, and if so, transmitting data migration related parameters to a nova component of the OpenStack cloud computing management platform, wherein the data migration related parameters comprise: the identification information of the first instance of the first volume, the identification information of the first volume, and the identification information of the destination volume included by the destination pool in the ceph cluster are mounted;
utilizing the nova component to check the data migration related parameters to check whether the data in the first volume can be migrated to the destination volume; wherein the check fails when the first volume is not uniquely mounted on an instance;
when the data migration related parameter passes verification, updating volume information of the virtual machine including the first volume by using the nova component, so that the state of the first volume is in unloading, and the state of the destination volume is mounted on the first instance;
utilizing the nova component to call QEMU (virtual operating system simulator) to synchronize the data in the first volume into the destination volume;
calling the QEMU by using the nova component, and merging the data which needs to be written into the destination volume during data synchronization into the destination volume;
updating the mount information of the first instance by using the nova component, and unloading the first volume from the first instance.
Further, the transmitting the data migration related parameters to the nova component of the OpenStack cloud computing management platform includes: calling an update method of nova-api through the novaclent, and transmitting data migration related parameters to the nova component;
the identification information of the first instance on which the first volume is mounted includes instance _ uuid, the identification information of the first volume includes old _ volume.id, and the identification information of the destination volume includes new _ volume.id.
Further, the checking the data migration related parameter to check whether the data in the first volume can be migrated to the destination volume includes:
judging whether the first volume is only mounted on one instance, if so, continuing the current process;
judging whether the running state of the destination volume with the identification information of the destination volume is normal or not, if so, continuing the current process;
judging whether the size of the available storage space of the destination volume is not smaller than the size of the used storage space of the first volume, if so, continuing the current process;
and when the first volume is judged to be only mounted on one instance, the running state of the target volume is normal, and the size of the storage space of the target volume is not smaller than that of the first volume, determining that the data migration related parameters pass the verification.
Further, the updating the volume information of the virtual machine including the first volume includes:
calling a circle, volume, api, begin _ staging method of the circle component to change the state of the first volume to staging;
and calling a circle.volume.api.delete method of the circle component to update a database of the circle component, and mounting the destination volume on the first instance.
Further, the using the nova component to call QEMU to synchronize the data in the first volume to the destination volume includes:
calling a drive-mirror (node-D target) method of QEMU (qcow 2 sync) by using the nova component, and copying all data in the first volume into the destination volume.
Further, the invoking the QEMU by using the nova component, and merging the data that needs to be written into the destination volume during data synchronization into the destination volume includes:
and calling a block-jobdeck device of the QEMU to job0 method by using the nova component to confirm the data synchronization operation, and merging real-time data which are cached in the overlay images during the data synchronization period and need to be written into the destination volume into a backing file of the destination volume.
Further, the mount information of the first instance includes: volume information of an Extensible Markup Language (XML) file of the first instance.
In a second aspect, the present invention provides a device for migrating data across spots in a same ceph cluster, where the device is configured to perform the method in any one of the first aspects, and the device includes:
the first processing unit is configured to, by using a shader component of an OpenStack cloud computing management platform, determine, for a first volume included in a first pool in a ceph cluster of data to be migrated, whether a state of the first volume is mounted, and if so, transmit a data migration related parameter to a nova component of the OpenStack cloud computing management platform, where the data migration related parameter includes: the identification information of the first instance of the first volume, the identification information of the first volume, and the identification information of the destination volume included by the destination pool in the ceph cluster are mounted;
the second processing unit is configured to check the data migration related parameter by using the nova component to check whether the data in the first volume can be migrated to the destination volume; wherein the check fails when the first volume is not uniquely mounted on an instance; when the data migration related parameter passes verification, updating volume information of the virtual machine including the first volume by using the nova component, so that the state of the first volume is in unloading, and the state of the destination volume is mounted on the first instance; calling QEMU by utilizing the nova component, and synchronizing the data in the first volume into the destination volume; calling the QEMU by using the nova component, and merging the data which needs to be written into the destination volume during data synchronization into the destination volume; updating the mount information of the first instance by using the nova component, and unloading the first volume from the first instance.
In a third aspect, the present invention provides a device for data cross-pool migration in a same ceph cluster, including: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine-readable program to perform the method of any of the above first aspects.
In a fourth aspect, the present invention provides a computer readable medium having stored thereon computer instructions which, when executed by a processor, cause the processor to perform the method of any of the first aspects above.
The invention provides a method and a device for data cross-pool migration in the same ceph cluster. When judging that the state of the first volume of the first pool in the ceph cluster is mounted by using the circle, transmitting the data migration related parameters to the nova; and verifying the parameter by using nova to verify whether the data in the first volume can be migrated to the destination volume, if so, updating the volume information of the virtual machine to enable the state of the first volume to be in unloading and the state of the destination volume of the destination pool in the cluster to be mounted on the first instance, calling QEMU for data migration, merging the data which needs to be written into the destination volume during migration into the QEMU, updating the mounting information of the first instance and unloading the first volume from the QEMU. The invention can perform data cross-pool migration on the mounted volume.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for migrating data across spots in a same ceph cluster according to an embodiment of the present invention;
fig. 2 is a flowchart of another method for migrating data in the same ceph cluster across pools according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a data cross-pool migration device in the same ceph cluster according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a method for migrating data across spots in a same ceph cluster, which may include the following steps:
step 101: utilizing a shader component of an OpenStack cloud computing management platform, judging whether the state of a first volume included by a first pool in a ceph cluster of data to be migrated is mounted, and if so, transmitting data migration related parameters to a nova component of the OpenStack cloud computing management platform, wherein the data migration related parameters comprise: and otherwise, executing a native type to perform data migration and ending the current flow.
Step 102: utilizing the nova component to check the data migration related parameters to check whether the data in the first volume can be migrated to the destination volume; wherein the check fails when the first volume is not uniquely mounted on an instance.
Step 103: when the data migration related parameter passes the verification, updating the volume information of the virtual machine including the first volume by using the nova component, so that the state of the first volume is in unloading, and the state of the destination volume is mounted on the first instance.
Step 104: and utilizing the nova component to call QEMU to synchronize the data in the first volume into the destination volume.
Step 105: and calling the QEMU by using the nova component, and merging the data which needs to be written into the destination volume during data synchronization into the destination volume.
Step 106: updating the mount information of the first instance by using the nova component, and unloading the first volume from the first instance.
The method for data cross-pool migration in the same ceph cluster provided by the embodiment of the invention comprises the following steps: when judging that the state of the first volume of the first pool in the ceph cluster is mounted by using the circle, transmitting the data migration related parameters to the nova; and verifying the parameter by using nova to verify whether the data in the first volume can be migrated to the destination volume, if so, updating the volume information of the virtual machine to enable the state of the first volume to be in unloading and the state of the destination volume of the destination pool in the cluster to be mounted on the first instance, calling QEMU for data migration, merging the data which needs to be written into the destination volume during migration into the QEMU, updating the mounting information of the first instance and unloading the first volume from the QEMU. The embodiment of the invention can perform data cross-pool migration on the mounted volume. In detail, ceph stores disks that can support different media, such as SATA (Serial ATA) hard disks, SAS (Serial attached SCSI) hard disks, and SSDs (Solid State Disk or Solid State Drive, or Solid State Disk), which can be further classified as SATA SSDs and NVMe storage systems (Non-Volatile Memory express).
In detail, a ceph cluster may have multiple spots, each spot is a logical isolation unit, and different spots may have completely different data processing modes.
In detail, one or more volumes may be included under each pool. Wherein a volume (or volume, or fileset) may be a recognizable data storage unit.
In the embodiment of the invention, data is migrated across the pool, and the data in any volume of any pool is generally migrated to any other volume of any other pool, so as to realize flexible operation on the data. For convenience of technical description, the former volume may be referred to as a first volume, and the latter volume may be referred to as a destination volume.
Referring to the foregoing step 101-106, the embodiment of the present invention provides a specific implementation logic for supporting real-time migration (or retype operation) across ceph pool for volume in a mounted state (or in-use state) in an Openstack environment. In particular, data migration across the pool may be implemented utilizing the circle component and the nova component of the OpenStack cloud computing management platform.
In detail, nova is a virtual server deployment and business computation module.
In detail, the render block storage is an essential component in the virtual infrastructure, and is a basis for storing the virtual machine image file and the data used by the virtual machine. Different volume types are usually corresponded in the circle, and the corresponding service products are a common cloud disk, a high-efficiency cloud disk, an SSD disk and the like.
Referring to the foregoing step 101-106, in order to implement data cross-pool migration, in step 101, a circle may be first used to determine whether the first volume is mounted, if so, the relevant parameters of data migration are transmitted to nova, and if not, the subsequent processes in the data cross-pool migration method according to the embodiment of the present invention may not be executed, and a native type may be directly executed as needed to perform data migration.
In order to accurately perform data migration across the pool, the data migration related parameters generally include at least three parameters: and mounting identification information of instance of the first volume, identification information of the first volume and identification information of the destination volume. Therefore, each object can be accurately positioned according to the corresponding relation between the object and the identification.
In detail, when the state of any volume is mounted, it indicates that the volume is currently mounted on at least one instance. In detail, an instance may be a series of processes in an operating system and a block of memory allocated for the processes.
In step 102, the data migration related parameters sent by the circle can be checked by the nova to determine whether the data in the first volume can be migrated to the destination volume. Step 103 may be performed when the verification passes, and the current process may be ended when the verification fails.
And if the first volume is mounted on at least two instances, the verification fails.
In step 103, the volume information of the virtual machine to which the first volume belongs may be updated by using the nova, so that the state of the first volume is in unloading, and the state of the destination volume is mounted on the first instance.
Since the verification passes, it is described that the identification information of the instance on which the first volume is mounted in the data migration related parameters is 1, so that the instance, that is, the first instance, can be determined according to the identification information, and the destination volume is mounted on the first instance.
After updating the volume information, in step 104, the QEMU may be called by nova to perform data migration, so as to migrate the data in the first volume to the destination volume.
In the process of executing data migration, there may be a case that other users write data into the destination volume, and the part of data is usually cached elsewhere, so after the data migration is completed, in step 105, QEMU may be called by nova, and the part of cached data is merged into the destination volume.
Finally, in step 106, the mount information of the first instance can be updated by using nova, and the first volume is unloaded from the first instance. Therefore, the real-time migration of data in the volume in the mounted state across ceph pool under the Openstack environment is completed.
In summary, based on the method for migrating data across pools in the same ceph cluster provided by the embodiment of the present invention, a user can switch between cloud disks according to a service, so as to implement migration and replication between pools without data loss, avoid the limitation of Openstack during data processing, and achieve good user experience.
In an embodiment of the present invention, in the step 101, the transmitting the data migration related parameters to the nova component of the OpenStack cloud computing management platform includes: calling an update method of nova-api through the novaclent, and transmitting data migration related parameters to the nova component; the identification information of the first instance on which the first volume is mounted includes instance _ uuid, the identification information of the first volume includes old _ volume.id, and the identification information of the destination volume includes new _ volume.id.
In detail, nova-api is an interactive interface through which an administrator can manage internal infrastructure and also provide services to users.
In the embodiment of the invention, the circular judges the state of the old _ volume, if the old _ volume is in the mount state, the update method of nova-api is called by the novaclent, and the parameters of instance _ uuid, old _ volume.id and new _ volume.id are transmitted.
Specifically, the old _ volume corresponds to the first volume, and the new _ volume corresponds to the destination volume, as follows.
In an embodiment of the present invention, in the step 102, the checking the parameter related to data migration to check whether the data in the first volume can be migrated to the destination volume includes: judging whether the first volume is only mounted on one instance, if so, continuing the current process; judging whether the running state of the destination volume with the identification information of the destination volume is normal or not, if so, continuing the current process; judging whether the size of the available storage space of the destination volume is not smaller than the size of the used storage space of the first volume, if so, continuing the current process; and when the first volume is judged to be only mounted on one instance, the running state of the target volume is normal, and the size of the storage space of the target volume is not smaller than that of the first volume, determining that the data migration related parameters pass the verification.
In the embodiment of the invention, the old _ volume is checked in nova, whether multiple instances are mounted in the old _ volume is judged, whether the state of the new _ volume is normal or not is judged, and the size of the new _ volume is required to be larger than or equal to that of the old _ volume to be migrated.
In an embodiment of the present invention, in step 103, the updating the volume information of the virtual machine including the first volume includes: calling a circle, volume, api, begin _ staging method of the circle component to change the state of the first volume to staging; and calling a circle.volume.api.delete method of the circle component to update a database of the circle component, and mounting the destination volume on the first instance.
In the embodiment of the invention, the nova updates the volume information of the server. Specifically, in the process of unloading the volume by the virtual machine, two functions can be mainly involved:
(1) the nova calls a circle, volume, api, begin _ detail method to change the state of the old _ volume to detail so as to prevent other nodes from executing mounting operation;
(2) the nova calls a circle volume api delete method to update the circle database, and meanwhile, the newly-built volume (namely new _ volume) is mounted.
In one embodiment of the present invention, the step 104 comprises: calling a drive-mirror (node-D target) method of QEMU (qcow 2 sync) by using the nova component, and copying all data in the first volume into the destination volume.
In the embodiment of the present invention, nova copies the content of the running whole disk to a new target, that is, synchronizes the data in old _ volume to new _ volume, by calling a method of drive-mirror (drive-mirror) of qemu, where node-D target is e.g., qcow2 sync is full jobid 0.
It can be seen that in this approach, sync is full, i.e., the content of the entire disk image chain is synchronized to the target. In one embodiment of the present invention, the sync _ top or sync _ none may be set according to different requirements of practical applications. When sync top, only the contents of the topmost disk image in the chain are synchronized to the target, and when sync none, only new writes are synchronized from that time on.
In detail, Libvirt is an open source API, daemon, and management tool for managing a virtualization platform. It can be used to manage KVM, Xen, VMware ESX, QEMU and other virtualization technologies. In one embodiment of the invention, qemu may be encapsulated with some specific methods by Libvirt, so that nova can invoke the method encapsulated in qemu.
In one embodiment of the present invention, the step 105 comprises: and calling a block-jobdeck device of the QEMU to job0 method by using the nova component to confirm the data synchronization operation, and merging real-time data which are cached in the overlay images during the data synchronization period and need to be written into the destination volume into a backing file of the destination volume.
In the embodiment of the invention, after the nova data are copied, a jobb-completedevice of qemu is called to confirm the operation, and the real-time data in overlay images are merged into backing files.
In an embodiment of the present invention, in the step 106, the mount information of the first instance includes: volume information of the XML file of the first instance.
In the embodiment of the invention, nova updates xml of instance and removes old _ volume from instance.
Based on the above, as shown in fig. 2, an embodiment of the present invention provides a method for migrating data in volume in a mounted state in the same ceph cluster in real time in an Openstack environment, where the method includes the following steps:
step 201: and judging whether the old _ volume is in a mount state or not by using the circular, if so, executing the step 202, otherwise, executing the native retype to perform data migration, and ending the current process.
Step 202: and calling the update method of nova-api by using the finder through novaclent, and transmitting parameters of instance _ uuid, old _ volume.id and new _ volume.id into the nova.
In detail, nova may determine old _ volume according to old _ volume.id, determine new _ volume according to new _ volume.id, and determine instance of currently mounted old _ volume according to instance _ uuid.
Step 203: and checking the old _ volume by using nova to judge whether the old _ volume mounts a plurality of instances, if so, ending the current flow, otherwise, executing the step 204.
Step 204: and judging whether the state of the new _ volume is normal or not by using nova, if so, executing the step 205, and otherwise, ending the current process.
Step 205: and judging whether the size of the new _ volume is larger than or equal to the size of the old _ volume to be migrated or not by using nova, if so, executing the step 206, and otherwise, ending the current flow.
Step 206: in the process of unloading the volume by using nova, the state of old _ volume is changed to detail by calling a circle, volume, api, begin _ detail method, and new _ volume is mounted to update the volume information and database information of the server.
In detail, the information of the database can be updated by the circular attach mounting the newly created volume and returning the information of connection _ info.
Step 207: the content of the entire disk of the running old _ volume is copied into the new target new _ volume by the method of nova calling drive-mirror of qemu (drive-D target e. qcow2 sync full jobid job 0).
Step 208: after the data replication is finished, confirming the operation by using a method that nova calls block-jobdeck-complete-job 0 of qemu, and merging the real-time data in overlay images into a backing file.
In the embodiment of the invention, the newly written data is temporarily stored during the data migration, and is merged into the new _ volume after the data migration is completed.
Step 209: the xml of the instance is updated with nova, and old _ volume is dismatch from the instance.
The embodiment of the invention provides a method for supporting real-time migration of ceph pool to the volume in the mounted state in Openstack, namely, the method can realize flexible operation on data to the type processing flow logic of the volume in the mounted state across the pool.
The information interaction, execution process and other contents in the embodiment of the present invention are based on the same concept as those in the other embodiments, and specific contents may refer to the description in this embodiment, which is not described herein again.
As shown in fig. 3, an embodiment of the present invention provides a device for migrating data across spots in a same ceph cluster, where the device is configured to perform any one of the foregoing methods, and the device may include:
the first processing unit 301 is configured to, by using a shader component of an OpenStack cloud computing management platform, determine, for a first volume included in a first pool in a ceph cluster of data to be migrated, whether a state of the first volume is mounted, and if yes, transmit a data migration related parameter to a nova component of the OpenStack cloud computing management platform, where the data migration related parameter includes: the identification information of the first instance of the first volume, the identification information of the first volume, and the identification information of the destination volume included by the destination pool in the ceph cluster are mounted;
the second processing unit 302 is configured to check, by using the nova component, the data migration related parameter to check whether data in the first volume can be migrated to the destination volume; wherein the check fails when the first volume is not uniquely mounted on an instance; when the data migration related parameter passes verification, updating volume information of the virtual machine including the first volume by using the nova component, so that the state of the first volume is in unloading, and the state of the destination volume is mounted on the first instance; calling QEMU by utilizing the nova component, and synchronizing the data in the first volume into the destination volume; calling the QEMU by using the nova component, and merging the data which needs to be written into the destination volume during data synchronization into the destination volume; updating the mount information of the first instance by using the nova component, and unloading the first volume from the first instance.
Because the information interaction, execution process, and other contents between the units in the device are based on the same concept as the method embodiment of the present invention, specific contents may refer to the description in the method embodiment of the present invention, and are not described herein again.
Embodiments of the present invention also provide a computer-readable medium storing instructions for causing a computer to perform a method for data migration across pools in a same ceph cluster as described herein. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.
In summary, the embodiments of the present invention have at least the following advantages:
1. in the embodiment of the invention, when the Cinder is used for judging that the state of the first volume of the first pool in the ceph cluster is mounted, the data migration related parameters are transmitted to nova; and verifying the parameter by using nova to verify whether the data in the first volume can be migrated to the destination volume, if so, updating the volume information of the virtual machine to enable the state of the first volume to be in unloading and the state of the destination volume of the destination pool in the cluster to be mounted on the first instance, calling QEMU for data migration, merging the data which needs to be written into the destination volume during migration into the QEMU, updating the mounting information of the first instance and unloading the first volume from the QEMU. The embodiment of the invention can perform data cross-pool migration on the mounted volume.
2. Based on the data cross-pool migration method provided by the embodiment of the invention, a user can switch between cloud disks according to a service, migration and copy between pools are realized under the condition that data is not lost, limitation on Openstack during data processing is avoided, and user experience is good.
It should be noted that not all steps and modules in the above flows and system structure diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
In the above embodiments, the hardware unit may be implemented mechanically or electrically. For example, a hardware element may comprise permanently dedicated circuitry or logic (such as a dedicated processor, FPGA or ASIC) to perform the corresponding operations. The hardware elements may also comprise programmable logic or circuitry, such as a general purpose processor or other programmable processor, that may be temporarily configured by software to perform the corresponding operations. The specific implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other similar elements in a process, method, article, or apparatus that comprises the element.
While the invention has been shown and described in detail in the drawings and in the preferred embodiments, it is not intended to limit the invention to the embodiments disclosed, and it will be apparent to those skilled in the art that various combinations of the code auditing means in the various embodiments described above may be used to obtain further embodiments of the invention, which are also within the scope of the invention.
Claims (10)
1. The method for migrating data in the same ceph cluster across the pool is characterized by comprising the following steps:
utilizing a shader component of an OpenStack cloud computing management platform, judging whether the state of a first volume included by a first pool in a ceph cluster of data to be migrated is mounted, and if so, transmitting data migration related parameters to a nova component of the OpenStack cloud computing management platform, wherein the data migration related parameters comprise: the identification information of the first instance of the first volume, the identification information of the first volume, and the identification information of the destination volume included by the destination pool in the ceph cluster are mounted;
utilizing the nova component to check the data migration related parameters to check whether the data in the first volume can be migrated to the destination volume; wherein the check fails when the first volume is not uniquely mounted on an instance;
when the data migration related parameter passes verification, updating volume information of the virtual machine including the first volume by using the nova component, so that the state of the first volume is in unloading, and the state of the destination volume is mounted on the first instance;
calling a virtual operating system simulator QEMU by using the nova component, and synchronizing the data in the first volume into the destination volume;
calling the QEMU by using the nova component, and merging the data which needs to be written into the destination volume during data synchronization into the destination volume;
updating the mount information of the first instance by using the nova component, and unloading the first volume from the first instance.
2. The method of claim 1,
the nova component for transmitting the data migration related parameters to the OpenStack cloud computing management platform comprises: calling an update method of nova-api through the novaclent, and transmitting data migration related parameters to the nova component;
the identification information of the first instance on which the first volume is mounted includes instance _ uuid, the identification information of the first volume includes old _ volume.id, and the identification information of the destination volume includes new _ volume.id.
3. The method of claim 1,
the verifying the data migration related parameter to verify whether the data in the first volume can be migrated to the destination volume includes:
judging whether the first volume is only mounted on one instance, if so, continuing the current process;
judging whether the running state of the destination volume with the identification information of the destination volume is normal or not, if so, continuing the current process;
judging whether the size of the available storage space of the destination volume is not smaller than the size of the used storage space of the first volume, if so, continuing the current process;
and when the first volume is judged to be only mounted on one instance, the running state of the target volume is normal, and the size of the storage space of the target volume is not smaller than that of the first volume, determining that the data migration related parameters pass the verification.
4. The method of claim 1,
the updating the volume information of the virtual machine including the first volume comprises:
calling a circle, volume, api, begin _ staging method of the circle component to change the state of the first volume to staging;
and calling a circle.volume.api.delete method of the circle component to update a database of the circle component, and mounting the destination volume on the first instance.
5. The method of claim 1,
the utilizing the nova component to call QEMU to synchronize the data in the first volume to the destination volume includes:
calling a drive-mirror (node-D target) method of QEMU (qcow 2 sync) by using the nova component, and copying all data in the first volume into the destination volume.
6. The method of claim 1,
the calling the QEMU by using the nova component, and merging the data which needs to be written into the destination volume during the data synchronization period into the destination volume, wherein the merging comprises:
and calling a block-jobdeck device of the QEMU to job0 method by using the nova component to confirm the data synchronization operation, and merging real-time data which are cached in the overlay images during the data synchronization period and need to be written into the destination volume into a backing file of the destination volume.
7. The method according to any one of claims 1 to 6,
the mount information of the first instance comprises: and the volume information of the extensible markup language XML file of the first instance.
8. A device for migrating data across spots in a same ceph cluster, for performing the method of any one of claims 1 to 7, comprising:
the first processing unit is configured to, by using a shader component of an OpenStack cloud computing management platform, determine, for a first volume included in a first pool in a ceph cluster of data to be migrated, whether a state of the first volume is mounted, and if so, transmit a data migration related parameter to a nova component of the OpenStack cloud computing management platform, where the data migration related parameter includes: the identification information of the first instance of the first volume, the identification information of the first volume, and the identification information of the destination volume included by the destination pool in the ceph cluster are mounted;
the second processing unit is configured to check the data migration related parameter by using the nova component to check whether the data in the first volume can be migrated to the destination volume; wherein the check fails when the first volume is not uniquely mounted on an instance; when the data migration related parameter passes verification, updating volume information of the virtual machine including the first volume by using the nova component, so that the state of the first volume is in unloading, and the state of the destination volume is mounted on the first instance; calling a virtual operating system simulator QEMU by using the nova component, and synchronizing the data in the first volume into the destination volume; calling the QEMU by using the nova component, and merging the data which needs to be written into the destination volume during data synchronization into the destination volume; updating the mount information of the first instance by using the nova component, and unloading the first volume from the first instance.
9. With data cross pool migration device in ceph cluster, its characterized in that includes: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor, configured to invoke the machine readable program to perform the method of any of claims 1 to 7.
10. A computer readable medium having stored thereon computer instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010090739.1A CN111338751B (en) | 2020-02-13 | 2020-02-13 | Cross-pool migration method and device for data in same ceph cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010090739.1A CN111338751B (en) | 2020-02-13 | 2020-02-13 | Cross-pool migration method and device for data in same ceph cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111338751A CN111338751A (en) | 2020-06-26 |
| CN111338751B true CN111338751B (en) | 2022-03-22 |
Family
ID=71181523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010090739.1A Active CN111338751B (en) | 2020-02-13 | 2020-02-13 | Cross-pool migration method and device for data in same ceph cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111338751B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131145B (en) * | 2020-09-08 | 2021-11-09 | 北京易捷思达科技发展有限公司 | Caching method and device applied to ceph |
| CN112764830B (en) * | 2021-01-19 | 2023-04-21 | 长沙证通云计算有限公司 | Data migration method and system applied to localization substitution |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9363190B2 (en) * | 2013-07-31 | 2016-06-07 | Manjrasoft Pty. Ltd. | System, method and computer program product for energy-efficient and service level agreement (SLA)-based management of data centers for cloud computing |
| CN106708748B (en) * | 2016-12-21 | 2020-05-26 | 南京富士通南大软件技术有限公司 | Method and system for improving mounting performance of OpenStack block storage volume |
| CN108268299A (en) * | 2016-12-29 | 2018-07-10 | 航天信息股份有限公司 | A kind of Docker based on OpenStack cloud platforms shares storage solution |
| CN109582234A (en) * | 2018-11-23 | 2019-04-05 | 金色熊猫有限公司 | Storage resource allocation method, device, electronic equipment and computer readable medium |
| CN110058813A (en) * | 2019-03-15 | 2019-07-26 | 启迪云计算有限公司 | It is a kind of that management method is locally stored based on the storage of cloud platform block |
| CN110750334B (en) * | 2019-10-25 | 2024-02-02 | 北京计算机技术及应用研究所 | Ceph-based network target range rear end storage system design method |
-
2020
- 2020-02-13 CN CN202010090739.1A patent/CN111338751B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111338751A (en) | 2020-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110286930B (en) | Cloud platform upgrading method and device, terminal and storage medium | |
| KR102779758B1 (en) | Asynchronous cross-region block volume replication | |
| CN109558215B (en) | Backup method, recovery method and device of virtual machine and backup server cluster | |
| US10216431B2 (en) | Consistent data retrieval in a multi-site computing infrastructure | |
| US8949182B2 (en) | Continuous and asynchronous replication of a consistent dataset | |
| US10698866B2 (en) | Synchronizing updates across cluster filesystems | |
| US9471365B2 (en) | Techniques for performing virtual machine software upgrades using virtual disk swapping | |
| CN118276783A (en) | Data partition switching between storage clusters | |
| US8495609B2 (en) | Methods, systems, and computer program products for taking a snapshot of installed software on a data processing system as part of a software update process | |
| JP6516730B2 (en) | Method and apparatus for backing up and restoring applications | |
| US20120137285A1 (en) | Planning a reliable migration in a limited stability virtualized environment | |
| CN104216801B (en) | The data copy method and system of a kind of Virtual environment | |
| CN110347483B (en) | Method and device for migrating physical machine to virtual machine and storage medium | |
| CN104918114A (en) | Method and device for upgrading operation system | |
| US10169027B2 (en) | Upgrade of an operating system of a virtual machine | |
| CN114816676A (en) | System and method for realizing multi-tenant deployment of low code development platform | |
| CN111338751B (en) | Cross-pool migration method and device for data in same ceph cluster | |
| CN116225617A (en) | Management migration method and device for container instance, electronic equipment and storage medium | |
| US9904600B2 (en) | Generating initial copy in replication initialization | |
| CN106484312A (en) | A kind of magnetic disk of virtual machine data migration method and device | |
| CN113934441B (en) | A cloud platform upgrade method, device, equipment and storage medium | |
| US11204940B2 (en) | Data replication conflict processing after structural changes to a database | |
| US12229577B2 (en) | Virtual machine file management using file-level snapshots | |
| US10585601B2 (en) | Information processing apparatus, storage apparatus, and information processing system wherein virtual volume to physical volume mapping is updated from management machine responsive to storage indicating said physical volume is not present | |
| CN120704797A (en) | Container group migration method, apparatus, device, storage medium, and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220301 Address after: 250100 No. 1036 Tidal Road, Jinan High-tech Zone, Shandong Province, S01 Building, Tidal Science Park Applicant after: Inspur cloud Information Technology Co.,Ltd. Address before: Floor S06, Inspur Science Park, No. 1036, Inspur Road, hi tech Zone, Jinan City, Shandong Province Applicant before: SHANDONG HUIMAO ELECTRONIC PORT Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |