Architecture for managing I/O and storage for a virtualization environment

US 8 863 124B1

drawing #0

Show all 22 drawings

Disclosed is an improved approach to implement I/O and storage device management in a virtualization environment. According to some approaches, a Service VM is employed to control and manage any type of storage device, including directly attached storage in addition to networked and cloud storage. The Service VM implements the Storage Controller logic in the user space, and can be migrated as needed from one node to another. IP-based requests are used to send I/O request to the Service VMs. The Service VM can directly implement storage and I/O optimizations within the direct data access path, without the need for add-on products.

PatentSwarm provides a collaborative workspace to search, highlight, annotate, and monitor patent data.

Start free trial Sign in

Tip: Select text to highlight, annotate, search, or share the selection.

Claims

1. A system for communicating to a storage controller in a virtualization environment, comprising:
a plurality of nodes, wherein the plurality of nodes implements a virtualization environment, and comprises a hypervisor and multiple user virtual machines;
a plurality of hardware storage devices that are accessed by the multiple user virtual machines; and
a storage controller implemented as a service virtual machine on each node of the plurality of nodes to manage access by the user virtual machines to the plurality of storage devices;
wherein each service virtual machine is dedicated to its corresponding node;
wherein a request for storage managed by a service virtual machine at another node is handled by the service virtual machine at the other node;
wherein for a node of the plurality of nodes, a first virtual switch is employed to handle internal network traffic with the service virtual machine on the node and a second virtual switch is employed to handle external network traffic with the service virtual machine on the node.

Show 18 dependent claims

Description

The present application is a continuation in part of application Ser. No. 13/207,345, filed on Aug. 10, 2011, entitled ARCHITECTURE FOR MANAGING I/O AND STORAGE FOR A VIRTUALIZATION ENVIRONMENT, and is also related to application Ser. No. 13/207,357, filed on Aug. 10, 2011, entitled METADATA FOR MANAGING I/O AND STORAGE FOR A VIRTUALIZATION ENVIRONMENT, application Ser. No. 13/207,365, filed on Aug. 10, 2011, entitled METHOD AND SYSTEM FOR IMPLEMENTING A MAINTENANCE SERVICE FOR MANAGING I/O AND STORAGE FOR A VIRTUALIZATION ENVIRONMENT, application Ser. No. 13/207,371, filed Aug. 10, 2011, entitled METHOD AND SYSTEM FOR IMPLEMENTING WRITABLE SNAPSHOTS IN A VIRTUALIZED STORAGE ENVIRONMENT, and application Ser. No. 13/207,375, filed Aug. 10, 2011, entitled METHOD AND SYSTEM FOR IMPLEMENTING A FAST CONVOLUTION FOR COMPUTING APPLICATIONS, and which are all hereby incorporated by reference in their entirety.

FIELD

This disclosure concerns an architecture for managing I/O and storage devices in a virtualization environment.

BACKGROUND

A virtual machine or a VM refers to a specific software-based implementation of a machine in a virtualization environment, in which the hardware resources of a real computer (e.g., CPU, memory, etc.) are virtualized or transformed into the underlying support for the fully functional virtual machine that can run its own operating system and applications on the underlying physical resources just like a real computer.

Virtualization works by inserting a thin layer of software directly on the computer hardware or on a host operating system. This layer of software contains a virtual machine monitor or hypervisor that allocates hardware resources dynamically and transparently. Multiple operating systems run concurrently on a single physical computer and share hardware resources with each other. By encapsulating an entire machine, including CPU, memory, operating system, and network devices, a virtual machine is completely compatible with most standard operating systems, applications, and device drivers. Most modern implementations allow several operating systems and applications to safely run at the same time on a single computer, with each having access to the resources it needs when it needs them.

Virtualization allows one to run multiple virtual machines on a single physical machine, with each virtual machine sharing the resources of that one physical computer across multiple environments. Different virtual machines can run different operating systems and multiple applications on the same physical computer.

One reason for the broad adoption of virtualization in modern business and computing environments is because of the resource utilization advantages provided by virtual machines. Without virtualization, if a physical machine is limited to a single dedicated operating system, then during periods of inactivity by the dedicated operating system the physical machine is not utilized to perform useful work. This is wasteful and inefficient if there are users on other physical machines which are currently waiting for computing resources. To address this problem, virtualization allows multiple VMs to share the underlying physical resources so that during periods of inactivity by one VM, other VMs can take advantage of the resource availability to process workloads. This can produce great efficiencies for the utilization of physical devices, and can result in reduced redundancies and better resource cost management.

Data Centers are often architected as diskless computers (application servers) that communicate with a set of networked storage appliances (storage servers) via a network, such as a Fiber Channel or Ethernet network. A storage server exposes volumes that are mounted by the application servers for their storage needs. If the storage server is a block-based server, it exposes a set of volumes that are also called Logical Unit Numbers (LUNs). If, on the other hand, a storage server is file-based, it exposes a set of volumes that are also called file systems. Either way, a volume is the smallest unit of administration for a storage device, e.g., a storage administrator can set policies to backup, snapshot, RAID-protect, or WAN-replicate a volume, but cannot do the same operations on a region of the LUN, or on a specific file in a file system.

Storage devices comprise one type of physical resources that can be managed and utilized in a virtualization environment. For example, VMWare is a company that provides products to implement virtualization, in which networked storage devices are managed by the VMWare virtualization software to provide the underlying storage infrastructure for the VMs in the computing environment. The VMWare approach implements a file system (VMFS) that exposes storage hardware to the VMs. The VMWare approach uses VMDK files to represent virtual disks that can be accessed by the VMs in the system. Effectively, a single volume can be accessed and shared among multiple VMs.

While this known approach does allow multiple VMs to perform I/O activities upon shared networked storage, there are also numerous drawbacks and inefficiencies with this approach. For example, because the VMWare approach is reliant upon the VMFS file system, administration of the storage units occurs at a too-broad level of granularity. While the virtualization administrator needs to manage VMs, the storage administrator is forced to manage coarse-grained volumes that are shared by multiple VMs. Configurations such as backup and snapshot frequencies, RAID properties, replication policies, performance and reliability guarantees etc. continue to be at a volume level, and that is problematic. Moreover, this conventional approach does not allow for certain storage-related optimizations to occur in the primary storage path.

Therefore, there is a need for an improved approach to implement I/O and storage device management in a virtualization environment.

SUMMARY

Embodiments of the present invention provide an architecture for managing I/O operations and storage devices for a virtualization environment. According to some embodiments, a Service VM is employed to control and manage any type of storage device, including direct-attached storage in addition to network-attached and cloud-attached storage. The Service VM implements the Storage Controller logic in the user space, and with the help of other Service VMs in a cluster, virtualizes all storage hardware as one global resource pool that is high in reliability, availability, and performance. IP-based requests are used to send I/O request to the Service VMs. The Service VM can directly implement storage and I/O optimizations within the direct data access path, without the need for add-on products. The term Controller VM may be used throughout interchangeably in place of the term Service VM.

Further details of aspects, objects, and advantages of the invention are described below in the detailed description, drawings, and claims. Both the foregoing general description and the following detailed description are exemplary and explanatory, and are not intended to be limiting as to the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings illustrate the design and utility of embodiments of the present invention, in which similar elements are referred to by common reference numerals. In order to better appreciate the advantages and objects of embodiments of the invention, reference should be made to the accompanying drawings. However, the drawings depict only certain embodiments of the invention, and should not be taken as limiting the scope of the invention.

FIG. 1 illustrates an example architecture to implement I/O and storage device management in a virtualization environment according to some embodiments of the invention.

FIGS. 2A-D show alternate approaches to implement I/O requests according to some embodiments of the invention.

FIG. 3 illustrates the components of a Service VM according to some embodiments of the invention.

FIG. 4 illustrates the components of an I/O Director module according to some embodiments of the invention.

FIG. 5 shows the actions of Admission Control to implement QoS functionality according to some embodiments of the invention.

FIG. 6 illustrates a storage hierarchy according to some embodiments of the invention.

FIG. 7 illustrates an approach for implementing deduplication according to some embodiments of the invention.

FIG. 8 illustrates metadata for implementing I/O and storage device management in a virtualization environment according to some embodiments of the invention.

FIG. 9 illustrates shared vDisks according to some embodiments of the invention.

FIG. 10 illustrates shared-nothing vDisks according to some embodiments of the invention.

FIG. 11 shows a flowchart of an approach for handling movement of virtual machines for shared nothing vDisks according to some embodiments of the invention.

FIGS. 12A-E illustrate the approach of FIG. 11.

FIG. 13 shows a flowchart of an approach for handling failure of a Service VM for a shared vDisk according to some embodiments of the invention.

FIG. 14 is a block diagram of a computing system suitable for implementing an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS OF THE INVENTION

Embodiments of the present invention provide an improved approach to implement I/O and storage device management in a virtualization environment. According to some embodiments, a Service VM is employed to control and manage any type of storage device, including direct-attached storage in addition to network-attached and cloud-attached storage. The Service VM implements the Storage Controller logic in the user space, and with the help of other Service VMs in a cluster, virtualizes all storage hardware as one global resource pool that is high in reliability, availability, and performance. IP-based requests are used to send I/O request to the Service VMs. The Service VM can directly implement storage and I/O optimizations within the direct data access path, without the need for add-on products.

FIG. 1 illustrates an architecture for implementing storage management in a virtualization environment according to some embodiments of the invention. The architecture of FIG. 1 can be implemented for a distributed platform that contains multiple servers 100a and 100b that manages multiple-tiers of storage. The multiple tiers of storage include storage that is accessible through a network 140, such as cloud storage 126 or networked storage 128 (e.g., a SAN or storage area network). Unlike the prior art, the present embodiment also permits local storage 122/124 that is within or directly attached to the server and/or appliance to be managed as part of the storage pool 160. Examples of such storage include Solid State Drives (henceforth SSDs) 125 or Hard Disk Drives (henceforth HDDs or spindle drives) 127. These collected storage devices, both local and networked, form a storage pool 160. Virtual disks (or vDisks) can be structured from the storage devices in the storage pool 160, as described in more detail below. As used herein, the term vDisk refers to the storage abstraction that is exposed by a Service VM to be used by a user VM. In some embodiments, the vDisk is exposed via iSCSI (internet small computer system interface) or NFS (network file system) and is mounted as a virtual disk on the user VM.

Each server 100a or 100b runs virtualization software, such as VMware ESX(i), Microsoft Hyper-V, or RedHat KVM. The virtualization software includes a hypervisor 130/132 to manage the interactions between the underlying hardware and the one or more user VMs 102a, 102b, 102c, and 102d that run client software.

Citations

US 8,683,153 B1 - Iterating for deduplication
A method is used in iterating for deduplication. A collection of data is selected from a set of storage extents. The collection of data is...

US 2010 37,243 A1 - APPARATUS AND METHOD OF SUPPORTING PLURALITY OF OPERATING SYSTEMS
An apparatus is provided for supporting a plurality of operating systems by single hardware, wherein the plurality of operating systems include a server operating system...

US 2010 262,586 A1 - VIRTUAL MACHINE DATA REPLICATION
Disclosed is a method and system for selectively restoring file-level data from a disk image backup. In embodiments, a virtual machine backup may be performed...

US 2012 54,746 A1 - SYSTEM SOFTWARE INTERFACES FOR SPACE-OPTIMIZED BLOCK DEVICES
Interfaces to storage devices that employ storage space optimization technologies, such as thin provisioning, are configured to enable the benefits gained from such technologies to...

US 2012 79,229 A1 - DATA STORAGE OPTIMIZATION FOR A VIRTUAL PLATFORM
A method for storing data in virtual system is described. The method includes selecting virtual blocks in the virtual disk for storage of data based...

US 2011 320,556 A1 - Techniques For Migrating A Virtual Machine Using Shared Storage
Techniques for providing the ability to live migrate a virtual machine from one physical host to another physical host employ shared storage as the transfer...

US 2013 13,865 A1 - DEDUPLICATION OF VIRTUAL MACHINE FILES IN A VIRTUALIZED DESKTOP ENVIRONMENT
Techniques for deduplication of virtual machine files in a virtualized desktop environment are described, including receiving data into a page cache, the data being received...

US 2010 162,039 A1 - High Availability and Disaster Recovery Using Virtualization
There is disclosed apparatus and processes which address problems in the area of providing high availability and disaster recovery for computing systems and the data...

US 2012 5,307 A1 - STORAGE VIRTUALIZATION
A method of providing access to a plurality of different file systems implemented across a plurality of storage spaces comprises receiving a request for at...

US 2011 208,909 A1 - REDUCTION OF I/O LATENCY FOR WRITABLE COPY-ON-WRITE SNAPSHOT FUNCTION
According to one aspect of the invention, a method of controlling a storage system comprises storing data in a first volume in the storage system...

US 2012 158,674 A1 - Indexing for deduplication
Systems and methods of indexing for deduplication are disclosed. An example method includes providing a first table in a first storage and a second table...

US 2010 332,889 A1 - MANAGEMENT OF INFORMATION TECHNOLOGY RISK USING VIRTUAL INFRASTRUCTURES
Information Technology Risk to an organization is associated with a plurality of virtual machines (VMs) each running on a plurality of hosts, each host being...

US 2011 10,515 A1 - BACKUP OF VIRTUAL MACHINES USING CLONED VIRTUAL MACHINES
A system and method for creating a backup of a virtual machine running on a host computer is described herein. The system and method operate...

US 2009 222,542 A1 - VIRTUAL SYSTEM AND METHOD IN A VIRTUAL SYSTEM
A virtual system, which realizes virtual computing environments for performing communication processing, includes a user virtual machine which is provided as a virtual machine environment...

US 2007 156,955 A1 - Method and apparatus for queuing disk drive access requests
A method includes receiving a request to access a disk drive. The request has a size. The method further includes selecting a queue, based at...

US 2005 102,672 A1 - Non-platform-specific unique identifier generation
A method of generating a unique identifier without requiring platform-specific software, a computer readable medium embodying instructions for implementing the method, and a system therefor...

US 2012 144,229 A1 - VIRTUALIZED CLUSTER COMMUNICATION SYSTEM
A method includes executing, in each of a number of nodes of a cluster communication system, a specialized instance of an operating system privileged to...

US 2012 84,445 A1 - AUTOMATIC REPLICATION AND MIGRATION OF LIVE VIRTUAL MACHINES
Systems and methods are disclosed herein to automatically replicate and migrate live virtual machine image (VM) files from a primary VM computing device to secondary...

US 2011 184,993 A1 - Independent Access to Virtual Machine Desktop Content
Methods, systems, and techniques for facilitating access to content stored remotely, for example, as part of a virtual machine infrastructure or elsewhere in a networked...

US 2011 154,318 A1 - VIRTUAL STORAGE TARGET OFFLOAD TECHNIQUES
A virtual machine storage service can be use a unique network identifier and a SR-IOV compliant device can be used to transport I/O between a...

US 2011 71,983 A1 - SERVER IMAGE MIGRATION
A system includes a first site having a first server and a first storage system, a second site including a second server and a second...

US 2012 272,240 A1 - Virtual Disk Storage Techniques
This document describes techniques for effecting a virtual disk. In an exemplary configuration, a virtual disk file can be associated with a log that acts...

US 8,413,146 B1 - Hypervisor assisted single instance data access by multiple virtual machines
A data instance to be shared by multiple virtual machines is stored at a hypervisor level. A file system driver is provided to each virtual...

US 2012 30,676 A1 - Methods And Apparatus For Creating An Isolated Partition For A Virtual Trusted Platform Module
A data processing system isolates a virtual trusted platform module (vTPM) manager in the processing system from other management software in the processing system. In...

US 2010 138,827 A1 - HASHING STORAGE IMAGES OF A VIRTUAL MACHINE
Virtual machine images are transferred from a source storage location to a target storage location over a network. In one embodiment, a host at the...

US 2005 65,985 A1 - Organization of read-write snapshot copies in a data storage system
A file system maintains a series of read-only snapshot copies of a production file system. A read-write snapshot copy is created based on a selected...

US 2012 96,461 A1 - LOAD BALANCING IN MULTI-SERVER VIRTUAL WORKPLACE ENVIRONMENTS
Virtual workplace server software may perform load balancing functionality in a multi-server desktop virtualization system. One or more virtualization servers may receive and maintain common...

US 2006 123,062 A1 - Virtual file system
A virtual file system and method. The system architecture enables a plurality of underlying file systems running on various file servers to be “virtualized” into...

US 2008 270,564 A1 - Virtual machine migration
Virtual machine migration is described. In embodiment(s), a virtual machine can be migrated from one host computer to another utilizing LUN (logic unit number) masking....

US 2011 61,050 A1 - METHODS AND SYSTEMS TO PROVIDE PLATFORM EXTENSIONS FOR TRUSTED VIRTUAL MACHINES
Methods and systems to authenticate a privileged virtual machine (VM), such as a monitoring VM, at a computing platform. Once authenticated, the privileged VM may...

US 2008 263,407 A1 - VIRTUAL COMPUTER SYSTEM
To improve the maintainability of a virtual machine system, there is provided a mechanism capable of appropriately reporting a failure that occurs in a dedicated...

US 2013 61,014 A1 - SYSTEMS AND METHODS FOR MANAGEMENT OF VIRTUALIZATION DATA
Described in detail herein is a method of copying data of one or more virtual machines being hosted by one or more non-virtual machines. The...

US 2008 244,028 A1 - Synchronization and Customization of a Clone Computer
A computer implemented method for maintaining synchronization between a master computer disk and a clone disk that includes cloning the clone disk from the master...

US 2012 221,529 A1 - VIRTUAL MACHINE DATA REPLICATION
Disclosed is a method and system for selectively restoring file-level data from a disk image backup. In embodiments, a virtual machine backup may be performed...

US 2008 183,973 A1 - Snapshots in distributed storage systems
Embodiments include methods, apparatus, and systems for snapshots in distributed storage systems. One method of software execution includes using a version tree to determine what...

US 2011 145,916 A1 - METHODS AND SYSTEMS FOR PREVENTING ACCESS TO DISPLAY GRAPHICS GENERATED BY A TRUSTED VIRTUAL MACHINE
The methods and systems described herein provide for preventing a non-trusted virtual machine from reading the graphical output of a trusted virtual machine. A graphics...

US 2008 282,047 A1 - METHODS AND APPARATUS TO BACKUP AND RESTORE DATA FOR VIRTUALIZED STORAGE AREA
Information about management of virtualization (i.e. mapping between physical area/data/objects and logical area/data/objects) is provided by a storage system to a backup appliance (or common...

US 2010 70,725 A1 - SYSTEMS AND METHODS FOR MANAGEMENT OF VIRTUALIZATION DATA
Described in detail herein is a method of copying data of one or more virtual machines being hosted by one or more non-virtual machines. The...

US 2006 5,189 A1 - Systems and methods for voluntary migration of a virtual machine between hosts with common storage connectivity
The present invention is a system for and method of performing disk migration in a virtual machine environment. The present invention provides a means for...

US 7,035,881 B2 - Organization of read-write snapshot copies in a data storage system
A file server maintains a series of read-only snapshot copies of a production file system. A read-write snapshot copy is created based on a selected...

US 2011 145,534 A1 - EFFICIENT LOADING OF DATA INTO MEMORY OF A COMPUTING SYSTEM
A method for deploying one or more virtual machines on a host computing system is provided. The method comprises receiving mapping information from a data...

US 2009 259,759 A1 - TERMINAL DEVICE, NETWORK CONNECTION METHOD, AND COMPUTER READABLE MEDIUM HAVING PROGRAM STORED THEREIN
A virtual machine system including a user virtual machine for operating a user environment, and a service virtual machine for controlling the user virtual machine,...

US 2013 14,103 A1 - COMBINED LIVE MIGRATION AND STORAGE MIGRATION USING FILE SHARES AND MIRRORING
Migration of a virtual machine and associated files to a destination host may be performed. A source host may initiate establishment of a temporary network...

US 7,360,034 B1 - Architecture for creating and maintaining virtual filers on a filer
An architecture provides the ability to create and maintain multiple instances of virtual servers, such as virtual filers (vfilers), within a server, such as a...

US 8,194,674 B1 - System and method for aggregating communications and for translating between overlapping internal network addresses and unique external network addresses
A gateway device including an access interface to an external network, a trunked interface, and translation logic. The access interface is associated with multiple external...

US 2011 258,404 A1 - METHOD AND APPARATUS TO MANAGE GROUPS FOR DEDUPLICATION
A storage system comprises one or more pool volumes having chunks for storing data; one or more primary volumes; writable snapshots as virtual volumes for...

US 2010 161,908 A1 - Efficient Memory Allocation Across Multiple Accessing Systems
Various embodiments of the present invention provide systems and methods for reducing memory usage across multiple virtual machines. For example, various embodiments of the present...

US 8,099,572 B1 - Efficient backup and restore of storage objects in a version set
The version set backup and restore facility responds to a version set backup request by backing up multiple snapshot copies of a storage object concurrently...

US 2010 281,166 A1 - Software Platform and System for Grid Computing
A software platform for providing grid computing on a network of computing nodes, comprising a configurable service container executable at the nodes, including message dispatching,...

US 2013 7,219 A1 - Shadowing Storage Gateway
Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway...

US 2010 251,238 A1 - FACILITATED INTROSPECTION OF VIRTUALIZED ENVIRONMENTS
Some embodiments provide a system that manages the execution of a software component in a virtualized environment. During operation, the system monitors the execution of...

US 8,407,265 B1 - Hierarchical mapping of free blocks of cylinder groups of file systems built on slices of storage and linking of the free blocks
In a file server, a file system is built upon a volume of data storage. The file system includes multiple cylinder groups. Each cylinder group...

US 2010 70,470 A1 - INFORMATION SYNCHRONISATION
A method of synchronising at least first and second instances of an information set comprising a plurality of entries, wherein said first and second instances...

US 8,566,821 B2 - Cloning virtual machines
While current solutions for cloning virtual machines can involve copying and duplicating files associated to a virtual machine (VM), systems and techniques can be devised...

PatentSwarm provides a collaborative workspace to search, highlight, annotate, and monitor patent data.

Start free trial Sign in