Live migration with pre-opened shared disks

US 9 588 796B2

drawing #0

Show all 11 drawings

Examples perform live migration of virtual machines (VM) from a source host to a destination host. The live migration performs time-consuming operations before the source host is stunned, reducing the downtime apparent to users. Some examples contemplate pre-copying memory from the source VM to the destination VM, and the opening of disks on the destination VM before stunning the source VM.

PatentSwarm provides a collaborative workspace to search, highlight, annotate, and monitor patent data.

Start free trial Sign in

Tip: Select text to highlight, annotate, search, or share the selection.

Claims

1. A method comprising:
receiving a request to perform a live migration of a source virtual machine (VM) on a source host to a destination VM on a destination host;
in response to the received request, downgrading a lock on at least one disk of the source VM from an exclusive mode to a non-exclusive mode;
opening the disk on the destination VM in the non-exclusive mode while the source VM is executing;
suspending execution of the source VM;
transferring a virtual device state of the source VM to the destination VM;
executing the destination VM;
closing the disk on the source VM; and
upgrading the lock on the disk by the destination VM from the non-exclusive mode to the exclusive mode.

Show 6 dependent claims

8. One or more non-transitory computer-readable storage media including computer-executable instructions that, when executed, cause at least one processor to perform live migration using shared disk locks by:
downgrading locks on one or more disks of a source virtual machine (VM) from an exclusive mode to a non-exclusive mode, wherein a destination VM opens the disks in non-exclusive mode while the source VM is executing;
suspending execution of the source VM;
transferring virtual memory of the source VM to the destination VM, wherein the destination VM begins execution after restoration of the virtual memory at the destination VM; and
closing the disks on the source VM, wherein the destination VM upgrades the locks from the non-exclusive mode to the exclusive mode.

Show 6 dependent claims

15. A system for performing live migration with one or more shared disks, said system comprising:
a memory area associated with a computing device, said memory area storing a destination virtual machine (VM); and
a processor programmed to:
open, in a non-exclusive mode on the destination VM, one or more disks associated with a source VM while the source VM is executing;
execute the destination VM after receiving a virtual device state of the source VM; and
after the source VM closes the disks, change the disks from the non-exclusive mode to an exclusive mode.

Show 5 dependent claims

Description

This application claims the benefit of U.S. Provisional Patent Application Ser. No. 62/018,582, filed Jun. 28, 2014 entitled Live Migration with Pre-Opened Shared Disks, U.S. Provisional Patent Application Ser. No. 62/018,580, filed Jun. 28, 2014 entitled Using Active/Active Asynchronous Replicated Storage for Live Migration, and U.S. Provisional Patent Application Ser. No. 62/041,047, filed Aug. 23, 2014 entitled Live Migration of Virtual Machines with Memory State Sharing, all of which are incorporated by reference herein in their entireties.

This application is related to U.S. Non-Provisional Patent Applications entitled Using Active/Active Asynchronous Replicated Storage for Live Migration and Live Migration of Virtual Machines with Memory State Sharing, filed concurrently herewith, both of which are incorporated by reference herein in their entireties.

BACKGROUND

Some existing systems migrate virtual machines (VMs) from a source host computing device to a destination host computing device. For example, the vMotion process from VMware, Inc. moves live, running VMs from one host to another without any perceptible service interruption. During the existing process of closing disks and releasing locks at the source host computing device, and then reopening disk and acquiring locks at the destination host computing device, the VM downtime or switchover time (e.g., the time a VM is not executing guest instructions during vMotion) is noticeable by customers because their workloads are stalled for the duration of that disk ownership handoff.

Some existing methods have switchover times, end-to-end, typically taking less than one second. However, some systems have many more disks per VM, to the point where a single VM may have over 100 disks. Closing and opening 100 disks during the downtime is problematic, at least because it can cause switchover times to extend to 2-5 seconds or greater.

In some examples, the disks are file extents on a VM file system (VMFS) or network file system (NFS), with disk open operations involving little more than simply opening the flat files and taking locks. However, with the advent of virtual volumes (VVOLs) and virtual storage array network (vSANs), object-backed disks are now supported for live migration. With VVOL and vSAN, opening a disk is far more complex. For example, the host calls out to an external entity, such as a vendor provider (VP), to request that the particular object be bound to the host. A number of other calls flow back and forth between the host and VP to prepare and complete the binding process. Only after that communication finishes can locks of the disk be acquired. The disk open is then declared to have completed successfully. Opening a single VVOL or vSAN disk, then, may take a full second or greater thereby increasing the downtime and reducing switchover performance. Moreover, in this example, the switchover performance is now dependent on performance of code from the VP (e.g., to release and bind locks).

Some existing methods of optimizing disk handoff during switchover have involved prepopulating disk lookup information at the destination host, and/or use multiple threads to concurrently open disks. However, there is no guarantee that any number of concurrent requests will be handled in parallel.

Even with the existing methods of disk handoff and live migration, it is increasingly difficult to migrate more complicated and larger systems from a source VM to a destination VM without increasing VM downtime. Further, with some of the existing systems, the disks are maintained by VPs which creates uncontrollable or unknowable VM downtimes because of the partner code run by the VPs. This can create visible delays in processing during live migration that are unacceptable to users.

SUMMARY

One or more examples described herein enable live migration using shared disks of a source virtual machine (VM) to a destination VM. A processor opens one or more disks of the source VM in non-exclusive mode on the destination VM while the source VM is still executing, to enable pre-copy from the source VM to the destination VM. The processor executes the destination VM after receiving a virtual device state of the source VM, and the processor changes the disks on the destination VM from the non-exclusive mode to an exclusive mode after the source VM closes the disks.

To facilitate the live migration, some examples described herein downgrade the disk locks to multiwriter lock mode, thus permitting both the source VM and destination VM to access the disk. However, the destination VM is prevented from writing to the disks during the multiwriter lock mode.

This summary introduces a selection of concepts that are described in more detail below. This summary is not intended to identify essential features, nor to limit in any way the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary host computing device.

FIG. 2 is a block diagram of virtual machines that are instantiated on a computing device, such as the host computing device shown in FIG. 1.

FIG. 3 is an exemplary sequence of migration as performed by a source VM and a destination VM as known in the art.

FIG. 4 is an exemplary sequence of live migration as performed by a source VM and a destination VM with the disks being opened in multiwriter mode.

FIG. 5 is a block diagram of a system for performing live migration of a source VM to a destination VM involving shared disks.

FIGS. 6A and 6B are flowcharts of an exemplary method of live migration of a source VM to a destination VM as performed by a hypervisor.

FIG. 7 is a flowchart of an exemplary method of live migration of a source VM to a destination VM from the perspective of the source VM.

FIG. 8 is a flowchart of an exemplary method of live migration of a source VM to a destination VM from the perspective of the destination VM.

FIG. 9A is a block diagram of an exemplary disk lock structure for a network file system (NFS) or virtual machine file system (VMFS) volume.

FIG. 9B is a block diagram of an exemplary disk lock structure for a virtual volume (VVOL).

Corresponding reference characters indicate corresponding parts throughout the drawings.

DETAILED DESCRIPTION

Aspects of the disclosure present a live migration scheme that accommodates VMs having numerous disks 534, and accounts for potentially longer switchover times due to opening/closing those disks 534. In some examples, disk operations are performed while a source VM 506 is still running, which keeps the switchover time to a minimum. For example, rather than sequentially providing exclusive access to disks 534 involved in a live migration, aspects of the disclosure overlap shared access to the disks 534 (e.g., by the source VM 506 and a destination VM 526) which effectively moves expensive disk operations to outside a downtime window. Even though both the source VM 506 and the destination VM 526 share a writeable state to the disks 534, at least the destination VM 526 is prevented from writing to these disks 534 while sharing access. In some examples, the source VM 506 is also prevented from writing to these disks 534 at this time. This prevents corruption of the disks 534 and prevents the introduction of inconsistencies in the disks 534.

Shared access to the disks 534 is implemented by shared disk locks, multiwriter locking, and/or other means.

The live migration described herein improves the functioning of VMs, and provides continuity of service as a VM is migrated from one host to another. Further, VM downtime (or unavailability) is decreased during the live migration. In some examples, there is no noticeable delay for users in accessing the VM, thereby improving the human-machine interface experience.

Examples of the modified live migration process are described as follows, although variations in the operations to achieve similar results are within the scope of the disclosure. Further, while the operations are sequentially ordered in the figures (e.g., FIG. 4), flexibility in the order of execution of the operations is within the scope of the disclosure so long as at least some of the expensive (e.g., time-consuming) disk operations are removed from the switchover time window. Further, in an example, the operations are performed in response to a request for live migration of the source host 502 to a destination host 522.

Exemplary operations associated with live migration are described in the context of a virtualized environment, such as next described with reference to FIG. 1.

FIG. 1 is a block diagram of an exemplary host computing device 100. Host computing device 100 includes a processor 102 for executing instructions. In some examples, executable instructions are stored in a memory 104. Memory 104 is any device allowing information, such as executable instructions and/or other data, to be stored and retrieved. For example, memory 104 includes one or more random access memory (RAM) modules, flash memory modules, hard disks, solid-state disks, and/or optical disks. In FIG. 1, memory 104 refers to memory and/or storage. However, in some examples, memory 104 refers only to memory in host computing device 100, and exclude storage units such as disk drives and hard drives. Other definitions of memory are contemplated.

Host computing device 100 may include a user interface device 110 for receiving data from a user 108 and/or for presenting data to user 108. User 108 may interact indirectly with host computing device 100 via another computing device such as VMware's vCenter Server or other management device. User interface device 110 may include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad or a touch screen), a gyroscope, an accelerometer, a position detector, and/or an audio input device. In some examples, user interface device 110 operates to receive data from user 108, while another device (e.g., a presentation device) operates to present data to user 108. In other examples, user interface device 110 has a single component, such as a touch screen, that functions to both output data to user 108 and receive data from user 108. In such examples, user interface device 110 operates as a presentation device for presenting information to user 108. In such examples, user interface device 110 represents any component capable of conveying information to user 108. For example, user interface device 110 may include, without limitation, a display device (e.g., a liquid crystal display (LCD), organic light emitting diode (OLED) display, or electronic ink display) and/or an audio output device (e.g., a speaker or headphones). In some examples, user interface device 110 includes an output adapter, such as a video adapter and/or an audio adapter. An output adapter is operatively coupled to processor 102 and configured to be operatively coupled to an output device, such as a display device or an audio output device.

Host computing device 100 also includes a network communication interface 112, which enables host computing device 100 to communicate with a remote device (e.g., another computing device) via a communication medium, such as a wired or wireless packet network. For example, host computing device 100 may transmit and/or receive data via network communication interface 112. User interface device 110 and/or network communication interface 112 may be referred to collectively as an input interface and may be configured to receive information from user 108.

Citations

US 9,251,004 B1 - System and method for application isolation with live migration
A system, method, computer program, and/or computer readable medium for providing application isolation and live migration to one or more applications and their associated resources....

US 2005 289,143 A1 - Method for managing lock resources in a distributed storage system
A method and a distributed locking system for the management of lock resources in a distributed files system having several lock managers, each managing lock...

US 2010 64,144 A1 - DATA SECURITY
This document discloses data security systems and methods of securing data. A cache memory can be connected between a decryption engine and a central processing...

US 2008 127,182 A1 - Managing Memory Pages During Virtual Machine Migration
A method, system and computer-readable medium is presented for migrating a virtual machine, from a first computer to a second computer, in a manner that...

US 2009 63,751 A1 - METHOD FOR MIGRATING CONTENTS OF A MEMORY ON A VIRTUAL MACHINE
A method for migrating contents of a memory on a virtual machine. The method includes computer instructions for establishing communication from a source virtual machine...

US 2012 233,285 A1 - STORAGE MEDIUM, INFORMATION PROCESSING APPARATUS, AND MIGRATION METHOD
A computer-readable storage medium stores program code for causing a computer to perform a method. The method includes steps of creating a redundant configuration including...

US 2011 321,041 A1 - METHOD AND SYSTEM FOR MIGRATING A VIRTUAL MACHINE
An exemplary embodiment of the present invention provides a method of migrating a virtual machine. The method comprises replicating a source storage device corresponding to...

US 2010 281,273 A1 - System and Method for Processor-Based Security
A system and method for processor-based security is provided, for on-chip security and trusted computing services for software applications. A processor is provided having a...

US 2012 284,234 A1 - LIVE MIGRATION OF VIRTUAL MACHINE PERSISTENT DATA USING MIRRORED INPUT-OUTPUT OPERATIONS
Persistent data, such as disks associated with virtual machines, are migrated between and across data stores using mirrored input-output operations (IOs). The migration technique employs...

US 2007 168,284 A1 - Management of encrypted storage media
A method for use of a physical data storage medium, the method including receiving a first read request for data stored in any of a...

US 2016 191,332 A1 - MANAGEMENT INFRASTRUCTURE ANALYSIS FOR CLOUD MIGRATION
In a source computing system having a source management infrastructure, at least one source infrastructure management component is discovered. A description of a target cloud...

US 2013 247,020 A1 - UPGRADE OF SOFTWARE IMAGES BASED ON STREAMING TECHNIQUE
Releasing a portion of mass memory, relocating memory blocks stored in boot locations of the target software image to the released portion, storing the boot...

US 2011 99,318 A1 - Leveraging Memory Similarity During Live Migrations
A page scanner may be configured to identify, during a live migration of a virtual machine including a transfer of a plurality of memory pages...

US 2013 42,241 A1 - DYNAMIC VIRTUAL MACHINE DOMAIN CONFIGURATION AND VIRTUAL MACHINE RELOCATION MANAGEMENT
Dynamic relocation of virtual machines among hypervisors in a domain is provided. A domain management module is executed on a computer processor and is configured...

US 2013 31,069 A1 - ALLOWING WRITES TO COMPLETE WITHOUT OBTAINING A WRITE LOCK TO A FILE
Provided are a computer program product, system, and method for allowing writes to complete without obtaining a write lock to a file. A lock is...

US 2012 284,707 A1 - METHOD AND SYSTEM FOR MIGRATING A SELECTED SET OF A VIRTUAL MACHINES BETWEEN VOLUMES
A method for migrating a selected set of virtual machines from a first volume to a second volume. The method includes receiving a list of...

US 2014 115,161 A1 - Method and system for migration of multi-tier virtual application across different clouds hypervisor platforms
A method and system for migration of multi-tier virtual application across different clouds hypervisor platforms. The principal object of the embodiment is to provide migration...

US 2015 212,839 A1 - TRACKING TRANSFORMED MEMORY PAGES IN VIRTUAL MACHINE CHAIN MIGRATION
Systems and methods for tracking transformed memory pages in virtual machine chain migration. An example method may comprise: receiving, by a computer system, a transformed...

US 2015 254,424 A1 - SYSTEM AND APPARATUS FOR IMPROVING MEDICATION REGIMEN ADHERENCE
A computer-based system for improving medication regimen adherence, including: a container arranged to hold a first medication and including a wireless transceiver and a unique...

US 2015 278,041 A1 - VM AVAILABILITY DURING MANAGEMENT AND VM NETWORK FAILURES IN HOST COMPUTING SYSTEMS
Techniques for virtual machine (VM) availability during management network failure and VM network failure in a first host computing system in a failover cluster are...

US 2015 205,542 A1 - VIRTUAL MACHINE MIGRATION IN SHARED STORAGE ENVIRONMENT
A source virtual machine (VM) executing on a source host is migrated to a destination host using a shared storage system connected to both hosts....

US 2014 310,800 A1 - SECURE DISK ACCESS CONTROL
A request is received from a security tool, the request relating to an event involving data records in a storage device. An application programming interface...

US 2016 55,019 A1 - METHOD AND SYSTEM FOR MIGRATING VIRTUAL MACHINES IN VIRTUAL INFRASTRUCTURE
A method of migrating a data compute node (DCN) in a datacenter comprising a plurality of host physical computing devices, a compute manager, and a...

US 2015 370,660 A1 - USING STRETCHED STORAGE TO OPTIMIZE DISASTER RECOVERY
Exemplary methods, apparatuses, and systems include determining that at least a portion of a protected site has become unavailable. A first logical storage device within...

US 2014 189,672 A1 - METHOD, COMPUTER PROGRAM PRODUCT, AND SYSTEM FOR NON-BLOCKING DYNAMIC UPDATE OF STATICALLY TYPED CLASS-BASED OBJECT-ORIENTED SOFTWARE
Under the present invention, a method, computer program product, and system for non-blocking dynamic update of statically-typed class-based object-oriented software executing as byte-code on a...

US 2015 178,109 A1 - Provisioning Customized Virtual Machines without Rebooting
Embodiments provision and customize virtual machines (VMs), such as desktop VMs, without rebooting the desktop VMs. In response to a request to provision the VMs,...

US 2014 229,697 A1 - ONLINE VIRTUAL MACHINE DISK MIGRATION
A method for migrating a virtual machine disk (VM disk) from first physical storage to second physical storage while the virtual machine (VM) is running,...

US 2015 74,453 A1 - MIGRATING VIRTUAL MACHINES ACROSS SITES
A method for managing data swapping in a migration of virtual memory data in a virtual machine, where the data swapping is from a first...

US 9,043,640 B1 - System and method for event-driven live migration of multi-process applications
A system, method, and computer readable medium for asynchronous live migration of applications between two or more servers. The computer readable medium includes computer-executable instructions...

US 7,680,919 B2 - Virtual machine migration
A source virtual machine (VM) hosted on a source server is migrated to a destination VM on a destination server without first powering down the...

US 2016 105,378 A1 - METHOD AND SYSTEM FOR DATA SET MIGRATION OVER A CIRCUIT SWITCHING NETWORK
A method for data set migration, implemented in a network device is disclosed. The method includes receiving a request at a network device to migrate...

US 8,954,408 B2 - Allowing writes to complete without obtaining a write lock to a file
Provided are a computer program product, system, and method for allowing writes to complete without obtaining a write lock to a file. A lock is...

US 8,407,182 B1 - Systems and methods for facilitating long-distance live migrations of virtual machines
A computer-implemented method for facilitating long-distance live migrations of virtual machines may include: 1) identifying a request for a live migration of a virtual machine...

US 9,081,842 B1 - Synchronous and asymmetric asynchronous active-active-active data access
In one aspect, a method to provide synchronous and asymmetric active-active-active access to a first storage volume using a global cache coupled to a first,...

US 8,527,990 B1 - Systems and methods for migrating virtual machines
A computer-implemented method for migrating virtual machines may include: 1) identifying a request to migrate a virtual machine from a primary site to a secondary...

US 2014 68,127 A1 - SHARED LOCKING MECHANISM FOR STORAGE CENTRIC LEASES
A computing device receives a request from a host for a shared lock on a resource. The computing device obtains an exclusive lock on the...

US 2015 178,108 A1 - Fast Instantiation of Virtual Machines
Embodiments support instant forking of virtual machines (VMs) and state customization. Virtual device state and persistent storage of a child VM are defined based on...

US 9,201,802 B1 - Managing read/write locks for multiple CPU cores for efficient access to storage array resources
A technique for managing resources on a storage array having multiple CPU cores releases share locks on a resource by decrementing share lock counters associated...

US 8,966,211 B1 - Techniques for dynamic binding of device identifiers to data storage devices
Described are techniques for performing dynamic binding of device identifiers to data storage devices. A first device identifier assigned to an application on a host...

US 2013 290,661 A1 - COMBINED LIVE MIGRATION AND STORAGE MIGRATION USING FILE SHARES AND MIRRORING
Migration of a virtual machine and associated files to a destination host may be performed. A source host may initiate establishment of a temporary network...

US 8,335,899 B1 - Active/active remote synchronous mirroring
An active/active remote mirroring system, for example an active/active SRDF system, provides for remote mirroring between multiple storage volumes, e.g., a first storage device and...

US 8,875,134 B1 - Active/active storage and virtual machine mobility over asynchronous distances
A system for providing mobility of a virtual machine between a first site and a second site of an active/active system is disclosed. For a...

US 2014 344,395 A1 - OBJECT MIGRATION BETWEEN CLOUD ENVIRONMENTS
Technologies are generally described for a system for migrating an application between cloud environments such as datacenters. In some examples, an application hosted at a...

PatentSwarm provides a collaborative workspace to search, highlight, annotate, and monitor patent data.

Start free trial Sign in