Tag Archives: flash

Enterprise storage is in for an exciting few years

The next generation storage symposium

At the start of November I was lucky enough to be invited to the Next Generation Storage Symposium in San Jose, California as well as Storage Field Day #2 on the following two days. There were sessions from upcoming storage vendors as well as a keynote from well known analyst Robin Harris and some thought provoking panel discussions about next generation storage technologies. Having spent some time digesting the flood of information and ideas from this trip there are two trends which ’emerge out of the haze’ for me;

  1. Flash storage continues to rapidly disrupt the storage marketplace
  2. The desire for more scalable systems is driving changes in the architecture of enterprise storage

To people immersed in storage these trends are well known and have been covered increasingly over the last couple of years (flash has been in mainstream arrays in some form since 2008). I’ve written this article to consolidate my own thoughts rather than as the traditional end of year predictions – ‘if you can’t explain it you don’t understand it‘ is something I believe and one of the principal reasons I enjoy blogging. Despite my interest in storage these events made me realise I’ve taken my eye off the ball and I’m now playing catchup!

Most of the vendor and panel sessions at the conference concentrated on the flash aspects – tiered vs cache, should you use a hybrid or all flash array (or no array at all in Nutanix’s case) although there was also discussion of the scalability of various architectures and technologies. Flash’s key advantage is performance – when compared to spinning disk it’s orders of magnitude faster and much of the current innovation is in trying to overcome its other constraints of cost, lifecycle, form factor etc. It’s not just performance that’s driving the current industry changes however – the desire for greater scalability is driving storage from a centralised model to a more distributed architecture (as described very eloquently by Chris Evans).

Combined these factors imply a major shakeup in the storage industry – it’s going to be a fun few years!

Flash disrupts the marketplace

Unless you’ve been hiding under a very big rock you can’t have missed the mass market arrival of flash devices into almost every aspect of the market, both for consumers and enterprises. Continue reading Enterprise storage is in for an exciting few years

Home labs – a poor man’s Fusion-IO?

While upgrading my home lab recently I found myself reconsidering the scale up vs scale out argument. There are countless articles about building your own home lab and whitebox hardware but is there a good alternative to the accepted ‘two whiteboxes and a NAS’ scenario that’s so common for entry level labs? I’m studying for the VCAP5-DCD so while the ‘up vs out’ discussion is a well trodden path there’s value (for me at least) in covering it again.

There are two main issues with many lab (and production) environments, mine included;

  1. Memory is a bottleneck and doubly so in labs using low end hardware – the vCentre appliance defaults to 8GB, as does vShield Manager so anyone wanting to play with vCloud (for example) needs a lot of RAM.
  2. Affordable yet performant shared storage is also a challenge – I’ve used both consumer NAS (from 2 to 5 bays) and ZFS based appliances but I’m still searching for more performance.

In an enterprise environment there are a variety of solutions to these challenges – memory density is increasing (up to 512GB per blade in the latest UCS servers for example) and on the storage front SSDs and flash memory have spurred innovations in the storage battle. In particular Fusion-IO have had great success with their flash memory devices which reduce the burden on shared storage while dramatically increasing performance. I was after something similar but without the budget.

When I built my newest home lab server, the vHydra I used a dual socket motherboard to maximise the possible RAM (up to 256GB RAM) and used local SSDs to supplement my shared storage. This has allowed me to solve the two issues above – I have a single server which can host a larger number of VMs with minimal reliance on my shared storage. The concepts are the same as solutions like Fusion-IO aim to do in production environments but mine isn’t particularly scalable. In fact it doesn’t really scale at all – I’ll have to revert to centralised storage if I buy more servers. Nor does it have any resilience – the ESXi server itself isn’t clustered and the storage is a single point of failure as there’s no RAID. It is cheap however, and for lab testing I can live with those compromises. None of this is vaguely new of course – Simon Gallagher’s vTardis has been using these same concepts to provide excellent lab solutions for years. Is this really a poor man’s Fusion-IO? There’s nothing like the peformance and nothing like the budget but the objectives are the same but to be honest it’s probably a slightly trolling blog title. I won’t do it again. Promise! 🙂

If you’re thinking of building a home lab from scratch consider buying a single large server with local SSD storage instead of multiple smaller servers with shared storage. You can always scale out later or wait for Ceph or HDFS to elimate the need for centralised storage at all…

Tip: It’s worth bearing in mind the 32GB limit on the free version of ESXi – unless you’re a vExpert or they reinstate the VMTN subscription you’ll be stuck with 60 day eval editions if you go above 32GB (or buying a licence!).

Further Reading

Is performant a word? 🙂