Netapp and vSphere5 storage integration
I’ve seen a few blogposts recently about storage features in vSphere5 and plenty of forum discussions about the level of support from various vendors but none that specifically address the Netapp world. As some of these features require your vendor to provide plugins and integration I’m going to cover the Netapp offerings and point out what works today and what’s promised for the future.
Many of the vSphere5 storage features work regardless of your underlying storage array, including StorageDRS, storage clusters, VMFS5 enhancements (provided you have block protocols) and the VMware Storage Appliance (vSA). The following vSphere features however are dependent on array integration;
- VAAI (the VMware Storage API for Array Integration). If you need a refresher on VAAI and what’s new in vSphere v5 check out these great blogposts by Dave Henry - part one covers block protocols (FC and iSCSI), part two covers NFS. The inimitable Chad Sakac from EMC also has a great post on the new vSphere5 primitives.
- VASA (the VMware Storage API for Storage Awareness). Introduced in vSphere5 this allows your storage array to send underlying implementation details of the datastore back to the ESXi host such as RAID levels, replication, dedupe, compression, number of spindles etc. These details can be used by other features such as Storage Profiles and StorageDRS to make more informed decisions.
The main point of administration (and integration) when using Netapp storage is the Virtual Storage Console (VSC), a vCenter plugin created by Netapp. If you haven’t already got this installed (the latest version is v4, released March 16th 2012) then go download it (NOW account required). As well as the vCenter plugin you must ensure your version of ONTAP also supports the vSphere functionality – as of April 19th 2012 the latest release is ONTAP 8.1. You can find out more about its featureset from Netapp’s Nick Howell. As well as the core vSphere storage features the VSC enables some extra features;
- Simplified administration
- Monitor and fix storage alignment
- Space reclamation
- Datastore replication. Luke Reed from Netapp has done a good blogpost on using this to duplicate templates between sites.
These features are all covered in Netapp’s popular TR3749 (best practices for vSphere, now updated for vSphere5) and the VSC release notes.
Poor old NFS – no VAAI for you…
It all sounds great! You’ve upgraded to vSphere5 (with Enterprise or Enterprise Plus licensing), installed the VSC vCenter plugin and upgraded ONTAP to the shiny new 8.1 release. Your Netapp arrays are in place and churning out 1′s and 0′s at a blinding rate and you’re looking forward to giving vSphere some time off for good behaviour and letting your Netapp do the heavy lifting…..
Are you using block protocols and therefore VMFS? If so, happy days! Unfortunately not everyone is so lucky. Have a look at the table below and see if you can spot those less fortunate souls;
|vSphere version||VAAI Feature||Benefit||VMFS||NFS||Netapp reqs|
|4.1||Full copy||Faster cloning, provisioning and svMotion for VMs||yes||n/a||ONTAP 8.0.1|
|Block zeroing||Creation of eager zeroed thick disks is faster||yes||n/a|
|h/w assisted locking||Increases scalability of VMFS - more VMs per LUN due to shorter lock times||yes||n/a|
|5.0||Thin provisioning stun||Prevents VMs crashing if a VMFS volume fills up||yes||n/a|
|Thin provisioning block reclaim||More efficient space management||partial - see VMwareKB2007427||n/a|
|NFS full copy||Faster cloning, provisioning and svMotion for VMs||n/a||yes||VAAI plugin
ONTAP 8.1 for cluster mode
ONTAP 8.1.1 for 7-mode
|NFS extended stats||Ability for ESXi to 'understand' more about the underlying storage (thin provisioned, dedupe etc)||n/a||yes|
|NFS reserve space||Thick provisioned NFS volumes (enabling FT over NFS potentially)||n/a||yes|
That’s right. Those pesky NFS users are still suffering because VAAI for NFS still isn’t enabled on Netapp arrays.
UPDATE: 24th May 2012 – as you’ll see in the comments, Netapp have released the VAAI plugin for Cluster mode arrays, although as noted above you’ll need ONTAP 8.1 and VSC 4.0. Go to the NOW site to download it, and read Nick Howell’s comment at the end of the article for more info.
UPDATE: 5th September 2012 – Netapp have now released ONTAP 8.1.1 (NOW login required) so VAAI for NFS should be available. I’ve not upgraded my Netapp controllers yet – when I do I’ll confirm this is working as expected.
VAAI needs a vendor supplied plugin to enable the secret storage offloading sauce and as of April 21st 2012 (more than eight months after vSphere5′s release) Netapp have yet to publish the plugin. Come on Netapp, get with the game – EMC had their NFS plugin released by the end of last year (although not yet for the Isilon)! The release notes for the most recent VSC state that the NFS VAAI plugin is included but that’s not the case unfortunately. Apparently it’s going through VMware’s certification process and is due in a matter of weeks but don’t count your chickens. If you’re running in 7 mode (rather than the new Cluster Mode) not only will you need the plugin but you’ll also need ONTAP 8.1.1, which isn’t released yet. Details on installing the plug-in will be in Knowledge base article 3013414 once it’s released.
Netapp do publish guidance on using VAAI with their arrays in TR3886 but unfortunately it was last updated in November 2010, long before the release of vSphere5. The only features covered in that report are for the first release of VAAI which didn’t support NFS anyway.
When VAAI for NFS does become a reality you’ll benefit in various ways (as shown in this Netapp demo of VAAI for VMFS and NFS);
- Faster VM provisioning (and faster restores if using technologies such as SMVI which clone a VM as part of the restoration process)
- Potential to run VMware Fault Tolerance over NFS (eager thick disks are a prerequisite and I believe they’ll be enabled by VAAI’s ‘block zeroing’)
- Storage capacity savings (thin provisioned VMs using Flexclone)
- Reduced load on your ESX hosts (network usage is almost eliminated for various operations)
What about Profile Driven Storage?
So VAAI for NFS doesn’t work yet, what about Profile Driven Storage, officially known as VM Storage Profiles?
There are two ways to use VM Storage Profiles – user defined, and system defined. The system defined method relies on VASA to ‘surface’ the capabilities of the storage array, which in turn requires a plugin from your storage vendor. Unlike VAAI which is enabled by default you need to configure VASA by configuring your vendor’s plugin. When I started writing this blogpost I was all set to whinge that Netapp hadn’t released their VASA provider either, but on the 25th April it was finally released (thanks to Eric Shanks for the heads up). You can download it from the Netapp support area (NOW login required) along with the release notes and an admin guide. Requirements;
- ONTAP 7.3.4 or greater, ONTAP 8.0.1 or greater
- Windows 2008 64bit (EE or R2) to install the VASA provider (must NOT be on your vCenter server)
- vCenter 5.0 or greater (and Enterprise+ licensing)
- Only 7-mode is supported (not cluster mode)
- vCenter Heartbeat and Metrocluster are not supported (but I guess this won’t affect most customers)
Eric has done a blogpost describing how to use it and a brief overview of benefits and Mike Laverick posted some good information about Storage Profiles (using the beta VASA provider). If you want to know more about Storage Profiles Cormac Hogan’s post covers user defined profiles (this post is not Netapp specific). With the Netapp VASA provider you can also configure alarms, currently on either thin provisioned capacity or when a capability changes.
VASA and Storage Profiles are a great idea and definitely have potential, but right now they’re definitely a v1 release with plenty to improve;
- The predefined ‘capabilities’ are pretty basic – anything on a SAS disk is marked as ‘high performance’ regardless of the number of spindles for example.
- The GUI could so with some refinement (this is a VMware issue not Netapp obviously)
How about the VSC features?
Finally let’s look at the VSC features. To be fair there are loads of features which work very well today with NFS – datastore creation/deletion on a cluster, visibility into capacity usage from volume through to datastore, host configuration (number of NFS mounts) etc. Frustratingly one of the best new features, nondisruptive storage alignment only works with VMFS (although NFS is promised at a later date). Aaaarrrgh!!
To be fair both FC and iSCSI often benefit from new features first because they account for a greater percentage of customer deployments (according to Chad Sakac’s end of 2011 survey). I also know that Netapp have being doing lots of work on the ONTAP 8.1 release and cluster mode, both of which are probably more important to most customers. But as an NFS aficionado and evangelist it’s sure frustrating!
Once the plugin is released, your ONTAP version is upgraded to 8.1.1 and you’re on vSphere5 you should see some performance and feature improvements which make all the effort worthwhile. Of course there are always some constraints and circumstances which affect these features – check this blogpost for some great diagnostic steps around VAAI.
Interestingly at VMworld 2011 there was an intriguing session (VSP3205 under the Cloud Computing vSphere track) titled “VMware vStorage APIs for VM and Application Granular Data Management” which talks about future ideas for advanced integration between vSphere and storage arrays. Julian Wood has a quick write up about it as has Scott Lowe but if you think VAAI and VASA are cool, you ain’t seen nothin’ yet!
Netapp demo of VAAI for VMFS and NFS (no audio but still useful)
Using VAAI UNMAP (Cormac Hogan)