Tag Archives: docker

Thoughts on the Docker Datacenter training course (beta)

Summary: A good training course that succeeded in teaching me how to deploy and manage a simple Docker Datacenter environment, as well as giving me some insight into the product itself.

Earlier this month I was lucky enough to take the beta training for Docker’s recently released Datacenter product. Being a beta it was offered for free (thankyou Docker!) although spaces were limited due to the courses being instructor led. There were only nine students on my course so it was quite personal, and despite being marketed for those in the US the attendees were from multiple timezones – one guy was doing it at 3am local time – so there’s obviously demand for training.

The attendees were from mixed backgrounds – interestingly most identified themselves as DevOps engineers or operations whereas I expected more developers. Levels of experience with Docker also varied from beginner to expert but all were able to complete the training successfully. I should state that while I’m not a complete beginner I’m certainly no expert either when it comes to Docker and containers.

The training consisted of three sessions, each four hours long, set over consecutive weeks. The course was offered online which is what made it possible for those like me working in remote places. It was about 80% hands on and 20% lecture time and the prerequisites are pretty simple – know what Docker is, how to pull and run container images, know Linux fundamentals (ssh, scp, navigating folders) etc. The training itself was similar to the training available at DockerConEU which is to say it uses AWS instances and content publicly available on Github. This is nice as it lets you have a look at some of the content in advance (in case you want to prepare or read up on some topics as I did) or after the course as a recap. It might be self explanatory enough to work through on your own, but be aware some sections have prerequisites which you’d need to setup yourself (rather than being provided them via AWS instances as we were). I received an email a few hours in advance of the course with a link to the online conference along with details of my AWS instances.

The presentations were simple overviews of the Docker products along with some background concepts, such as microservices. There were done in Reveal.js (which is pretty slick and worth learning, tutorial here) and had been containerised, meaning you can quickly and easily download it and run it locally (which could be useful for those that want to explore further at a later date);

docker pull training/docker-present
docker run --ti --rm -v /var/run/docker.sock:/var/run/docker.sock training/docker-present -p 8080

The three sessions covered the following topics;

  1. Deploying UCP. This involved the following tasks;
    • deploy the UCP controller to the first node
    • add two further nodes as compute nodes
    • integrate UCP with DTR and test by pulling/pushing an image using both CLI and the UCP GUI (including certificate setup)
  2. Managing container workloads in UCP. This involved the following tasks;
    • add two nodes to an existing UCP controller
    • deploy several nginx containers and a tomcat instance using UCP (varying port mappings)
    • deploy a simple application using Docker Compose (still done via CLI, though integration is on roadmap). Used projects on GitHub. Cloned to local directory and then ran using Compose.
  3. User management and troubleshooting UCP. This involved the following tasks;
    • create multiple users and groups via UCP
    • assign rights to those users and groups in UCP
    • create containers and assign and test permissions (using labels) in UCP
    • create containers via Docker Compose & using labels (CLI)
    • configure UCP to authenticate via an LDAP directory
    • reset the ‘admin’ account password

Each course begins and ends with a candidate assessment using Trueability.com, the idea being that you can measure your learning via the course. I like the idea but either due to product issues or Trueability idiosyncrasies the assessments didn’t really seem to work too well. I’m sure those issues will be ironed out by the time the courses are finalised.

Thoughts on the training and the product

The training content was good, as was the instructor, but the product felt quite basic and raw, which I guess is to be expected for a beta course on a new product. UCP seems to be a thin UI built on top of the Docker Engine/Swarm/Compose trio, and its integration could be more comprehensive. For instance;

  • docker-compose commands must still be run from the CLI (either locally or via an authenticated remote client), although the running containers are shown in the UI.
  • I and others got ‘Internal Server error 500’ when deploying a two container app via docker-compose. Rerunning the command a second time worked. The instructor had seen this behaviour previously and had fed info back to the developers as a bug but it’s not very reassuring given the simplicity of the task on a newly built infrastructure.
  • browser support seemed finicky – Chrome didn’t work with UCP week 1 whereas Safari did, but I got the opposite in week two
  • setting up overlay networking was also finicky and failed for several people. Creating networks in UCP is not refined – you have to type ‘overlay’ for network type instead of just a dropdown of available choices for instance
  • the public docs were outright wrong in a few places
  • deleting a network silently failed, simply because an app was defined as using it. There were no warnings, although the instructor was good at picking up these things.
  • when a user with insufficient permissions tries to pull a container image in UCP it simply hangs, rather than giving ‘permission denied’ or an error message
  • the version of the underlying Docker components was also  quite picky – several spurious errors were finally tracked down to running with a slightly old version of Docker Engine (1.9.1 rather than 1.10) and likewise with Docker Compose.

There’s a useful multi-container app in the Docker Github repository which was used as our trial app when using Docker Compose, but there’s nothing stopping you using it anytime you like. I really liked the cool SF FoodTruck app which we deployed during the course and it’s freely available on Github and was a good demonstration of how effective this method of deployment can be.

The SF FoodTruck app – great demo!

The post course assessment wasn’t really working but promises to be a nice feature – at least until formal certification becomes available. In general post course assessments aren’t worth having – either you can do the job or you can’t. Seriously though, how long can it be before you can be Docker Certified, maybe a Stevedor? 😉

In reality 12 hours of training, including lab work, isn’t going to cover much and there’s tons of complexity which we didn’t touch. I’m hoping to spend some more time with the product and experiment with some additional features, which I’ll write up here if I find the time.

As always I’d love to hear from anyone else with experience of Docker Datacenter. What do you think of it? Let me know in the comments or on Twitter.

My DockerCon Europe agenda

I’ve spent quite a bit of time recently familiarising myself with the ‘cloud native’ landscape (blog posts are in draft format and will hopefully get finished one day soon) and to that end I’m attending DockerCon Europe next week to immerse myself even further.  I’m going to be a fish out of water!  I’m used to attending VMworld where I know the technology and a large number of people whereas I only know a couple of people attending and only have a passing familiar with many of the technologies and vendors. Given the energy around the container space I’m envisaging an overwhelming experience like my first VMworld six years ago. You can view my full agenda online but there are a couple of sessions I’m particularly keen to attend.

The first is ‘cgroups, namespaces, and beyond: what are containers made from?‘ which promises a deep dive into container structure along with a comparison of different container runtimes. I’ve read about alternatives LXC, rkt, & systemd-nspawn a bit  and it’s particularly relevant to me as VMware’s container initiative, vSphere Integrated Containers, goes about creating the container using hypervisor mechanisms rather than Linux kernel primitives. Being able to articulate the rationale and benefits of this to VMware customers will be key. Incidentally for those who attended VMworld this year there was a great session all about it – INF5229 – Docker and Fargo: Exploding the Linux Container Host.

Another session which should have some lively debate is live migrating a container: pros, cons, and gotchas (a concept I tweeted about back in April). The obvious question is always ‘why would you want to do that? containers are stateless and immutable – deploy and destroy!’ but the naysayers would claim that containers will also be used to deploy stateful applications in which case this is a must have feature (just as vMotion is nowadays). The presenter is the maintainer of the CRIU project and also works for Odin whose product manages both VMs and containers (so they have both expertise and a desire for it to work!).

Hopefully I’ll follow up with a blogpost detailing my thoughts after the event. If you’re attending get in touch – it’d be nice to see a friendly face!

Reflections on VMworld 2014

Summary: A recap of the major announcement and my thoughts on both the announcements and the conference. It’s a long post because I use it as a personal record of thoughts – feel free to skim read!

Like last year I arrived in Barcelona on the Sunday so I had more time to settle in. This was my first conference as a VMware Partner but unfortunately Monday, Partner day, was a bit of a wash out for me due to some registration issues which preventing me getting into the sessions (and lunch!). I probably need to allow myself some time to adjust my perspective and learn the partner side of the fence and it’s unfair to judge when I didn’t attend but looking at session titles most of the partner sessions appeared to be sales focused rather than roadmap or vision which would have interested me more.  I guess everyone’s interested in those so they become general sessions. Which brings me nicely to the keynote presentations….

The keynotes

I’ve come to accept and almost enjoy the reality that Europe plays second fiddle to the US conference, which means the bulk of new announcement have already been made at the US show. My first ever blogpost was ranting about why the US show was the obvious one to attend but I now find I enjoy the gap as it gives me time to digest, investigate, and dwell on what’s new. It is a smaller show with less vendors, sessions etc but there’s still no way you can see or learn everything that’s on offer in the three or four days so it’s equally worth attending.

I think the buzz was a bit more balanced across the product suite this year. Two years ago felt like it was all about storage with the mass market adoption of caching, flash, scale out and hybrid arrays whereas last year was all about NSX. This year NSX was clearly still buzzing (top HOL by a mile) and storage continued it’s disruptive evolution (PernixData, VAIO, VSAN) but the announcement of EVO:RAIL got the most column inches. vCloud Air products, vRealize Automation adoption and some of the DevOps focus were also capturing plenty of the discussions and sessions. While VMware may be propping up ‘legacy’ applications until the Web 2.0/AWS crowd take over the world (;-)) it’s still a vibrant, exciting, and quick moving place – and therefore enjoyable!

I’ll recap the major announcements that caught my eye. When you look past the vRealize rebranding itself there are new releases – though most aren’t available until later this year (not too long to wait though);

  • SDDC (core infra)
    • EVORAIL smallEVO:RAIL (and later EVO:RACK) will allow VMware’s partners to compete with the existing hyperconverged vendors, while also selling more VMware licences.
    • vSphere 6 was NOT released but continues as an open beta. I’m on the beta and there are some great new features on the way (vSMP could be a game changer, vVOLs are an improvement but have taken too long to arrive) but this is somewhat overdue given VMware’s previous two year per major release lifecycle.
    • NSX 6.1 was announced (and released). NSX continues to grab mindshare but I think it’s going to be a long adoption cycle (as I’ve written previously).
  • Hybrid cloud
    • vCloud Air continues to evolve at a rapid pace. New services such as vRealize Air Mobile and vRealize Air Automation are the first to be announced but more will no doubt show up in short order.
    • vRealize Automation is being released on an aggressive six monthly release cycle which everyone is struggling to keep up with but it reflects the importance VMware attach to this product.
  • dockerMisc
    • VMware purchased CloudVolumes and rebranded it to AppVolumes. I first came across this technology last year via CloudCast episode 87 which is worth a listen as background. Interesting stuff and one to watch.
    • Docker integration was announced (the cynics would say to keep the DevOps crowd happy).  I agree that containers and VMs complement each other but I think containers are still a threat to VMware  in some use cases – after all containers run on any hypervisor so they level the playing field somewhat and containers without VMware are largely free….

If you want them all Latoga Labs have posted a comprehensive list of announcements at the US conference and you can supplement it with this list of announcements at Barcelona.

Reading between the lines – my thoughts

VMware’s entry in the hyperconverged space is both a big event and a non-event. It’s big news because it will increase adoption of hyperconverged infrastructure, particularly in the SMB space, through increased awareness and because EVO:RAIL is backed by large vendors. It’s a non-event in that EVO:RAIL doesn’t offer anything new other than form factor – it’s standard VMware technologies and you could already get similar (some would say superior) products from the likes of Nutanix and Simplivity and others. I’ll be posting my (generally positive) thoughts on EVO:RAIL soon (now posted).

NSX is here to stay. A cutdown version, NSX Lite, looks set to become a core part of vSphere at some point in the future, probably towards the end of 2015 (my guess). It may not have mass market adoption yet but there’s a lot of interest and actual customer deployments. It’s already baked into vCloud Air and will be part of the EVO:RACK stack when it’s released. VMware are clearly ‘betting the business’ on NSX succeeding.

The introduction last year of VMware’s own vCloud Hybrid Service, now known as vCloud Air (part of this year’s rebrand) makes it clear that even VMware’s partners weren’t keeping up so VMware have decided to compete their own way and create a public cloud where they can integrate the latest and greatest on a schedule they control. For some partners this evolution may be a challenge in the long term (is there still enough scope for adding value?) but for now it seems the partner network is alive and well. vCloud Air feels already gets as much focus from VMware as the vSphere suite, despite being only a year old, so I imagine we’ll see this pushed even more in the future. Whether they can really compete with the big four (AWS, Azure, Google, Rackspace) is yet to be seen – for Cloud IaaS VMware are still in the ‘niche’ quadrant according to Gartner.

Keeping customers engaged over vCloud Director seems to be an after thought for VMware, despite the technology underpinning vCloud Air. I spoke to several colleagues at large service providers and all felt that vCD had a future and that VMware would help them transition to vCAC (the successor to the crown). For the first year many colleagues were actively involved with vCO/vCAC and coding workflows, though most also mentioned a steep learning curve and required changes to organisational structures that need to accompany adoption.

Competition is forcing VMware’s hand. The tagline for this year’s conference was ‘No Limit’ and the keynote was peppered with references to being ‘brave’ (and last year’s tagline was a not too dissimilar ‘Deny convention’). I think VMware are trying to encourage their customers to accelerate their pace of adoption and change. In 2012 I wrote about customers struggling to keep up and still think it’s a problem today. Likewise competition is forcing VMware to release products throughout the year rather than at the conference. Five years ago everything was released at VMworld whereas the last few major releases have come outside conference time – VSAN, vCHS, even the vSphere 6 beta. VMworld is still a great marketing platform but major releases can now arrive at any time of year.

I’ve not very familiar with OpenStack but VMware’s development of an OpenStack distribution feels like they’re hedging their bets. If OpenStack adoption increases VMware have a stake in it and if not then there’s less competition. Time (and more educated folks) will tell.

This year I noted more of my peers developing code (for vRealize Automation – OK, vCAC) than ever before. vCenter Orchestrator has been ‘the best kept secret’ for about four years but the swing towards ‘infrastructure as code’ is actually taking hold in VMware-land.

Breakout Sessions/Hands On Labs

The HOLs were as popular as always

I only attended a few sessions this year – frankly I don’t know where the four days went! As all the sessions are online after the event I don’t prioritise them as much as I probably should, given that I rarely find time to watch them later! I was also more focused on work related technologies rather than new features as I was attending on company time rather than on my own time.

  • Site Recovery Manager 101: What’s New (BCO2394) – this was an introductory session which frankly I attended my mistake! I did learn a few new things and there was a nice tip at the end to check out another session (BCO1916.2 – Site Recovery Manager and Stretched Storage: Tech Preview of a New Approach to Active-Active Data Centers) which was covering a new SRM use case in combination with a MetroCluster. I didn’t have time to catch that session live but will be downloading it later.
  • Multi-Site Data Center Solutions with VMware NSX (NET1974 ). When I attended the NSX ICM course there was a lot of discussion around NSX being a single datacentre solution so I was curious what this session was going to cover. This was a great session, and covered both enterprise and cloud use cases and was surprisingly easy to digest for a complex topic – that’s the sign of a good speaker (Ray Badavari). Well worth a watch.
  • Veeam Availability suite v8 deep dive (STO2905-SPO). This session highlighted the new features in the upcoming v8 release along with some useful best practices. The failover plans look useful (very similar to SRM failover plans) and I can see a use case for SureReplica (test/dev sandbox for replica VMs) although many of the other features are just ‘nice to have’ rather than revolutionary – WAN acceleration for replication jobs (backup-copy jobs only in v7), network traffic encryption, Netapp integration etc.

I only found time for a few labs although I did find them useful (see also the full list of HOLs available in Barcelona). You can now take these labs online.

  •  HOL-SDC-1423 – vCloud Suite Networking. I need to improve my networking knowledge and getting more familiar with vCNS and NSX is high on my list. I found this lab hard going – not technically difficult, just boring! Note to self – don’t take labs at the end of a long day when tired as it’s not productive!
  • HOL-SDC-1428 – VMware EVO:RAIL Introduction. I enjoyed this lab, simple though it is. It gives you a chance to get hands on with the simple GUI available with the EVO:RAIL.
  • HOL-SDC-1429 – Virtual Volumes Tech Preview. This provided hands experience and helped me understand how actually administering vVol’s might work rather than just the theory of how they advance profile based management. Now we just have to wait for vSphere6 to be released…

The Solutions Exchange

chad at vmworld
Chad Sakacc pulls in the crowds

I didn’t make it to the vendor side of VMworld until late Wednesday and even then I spent less time than in previous years. The usual vendor enticements were on offer although I felt like the gimmicks and giveaways were slightly abated this year – less iPads, more t-shirts. This is a good thing, provided they can also provide technical information! I challenge myself every year to ‘take the pulse’ of the vendor ecosystem – who’s new, who’s thriving, who’s struggling, who’s going to be the next big thing? This year I thought the developments in the I/O path were one to watch so I checked out a few related vendors;

  • Pernix Data. I first saw these guys via Storage Field Day 3 back in April 2013 – that might not seem too long ago but in this industry it’s an age.Their ‘flash virtualisation platform’ is a read and write cache which operates across distributed hosts in a cluster to accelerate your I/O. I’ve also met with their CTO Satyam Vaghani on several occasions as I’m always impressed but both their technology and ambitions. Along with Proximal Data (who I saw at SFD2) these companies have been talking about the I/O path’s potential for a couple of years. There’s a reason it’s called a platform not just a product. Prior to VMworld this year PernixData launched v2.0 of their FVP platform which includes using RAM as a distributed cache. Good stuff!
  • Diablo Technologies. These guys deliver ultraDIMMs which essentially embed storage into your existing DIMM slots, facilitating blazing fast access in the process. In this thoughful introduction to Diablo by Justin Warren he tackles the technology and possible use cases. It’s certainly an interesting idea but are there enough use cases? I was hoping to see Diablo at TechFieldDay Extra but sadly they presented on the Thursday when I couldn’t attend – time to watch the videos I guess.
  • SanDisk. I spoke to FlashSoft (a division of SanDisk) back in 2012 when server side SANs were just getting started and PernixData etc were just coming out of stealth. Since then server-side caching has grown in popularity and this year SanDisk have partnered with VMware on the upcoming VAIO filters. FlashSoft never stuck me as the most popular flash cache solution so it’s interesting that VMware choose them as a partner. I wonder what this opening up of the APIs means to PernixData?

Here’s some further info and thoughts on some of these developments from Chris Wahl, Niels Hagoort, and Cormac Hogan. I was surprised that Proximal Data weren’t in attendance but their news page and twitter feed have been very quiet lately – maybe all is not well. I should also have spoken to Infinio and Atlantis Computing (as they operate in this space) but ran out of time.

For the first time (that I’m aware of) Oracle were in attendance. Given their licencing and certification/support stance on VMware it was brave to say the least! I’m familiar with their ‘converged’ infrastructure offering, the OVCA, having had some exposure to it at my previous employer but I didn’t find time to have a chat about it. I was surprised when Gartner put Oracle in the ‘leaders’ quadrant for converged infrastructure a few months ago (along with VCE, Netapp, and Cisco) but they must be doing something right. Whenever I mention OVM to anyone it gets short shrift though I’m not sure if that’s owing to actual knowledge or just because it’s ‘clever’ to bad mouth Oracle in the VMware world – personally I’ve never used OVM.

One of the interesting stands I always take time to visit is the VMware R&D team, now wrapped under the banner of ‘the office of the CTO’. They tend to pick a couple of ongoing projects and this year was no exception;

  • autoscaleAuto-scaling applications (Download the full PDF here).  I spoke to Xiaoyun Zhu who explained that they’re working on allowing applications to automatically scale out either vertically or horizontally based on a set of criteria. I remember trying to do something similar with dev/test environments and quickly found that while amending VMs is trivial that’s the tip of the iceberg – the application may need reconfiguring (buffers, caches etc) as will middleware and determining the ‘trigger’ for the initial memory upgrade is not always simple. How do you determine when to scale up vs out? What’s ‘typical’ performance for an app? What if only one tier in a multi-tier application needs to scale? What if scaling one tier has a knock on effect and you need to scale out every tier? The kind of machine learning used to create dynamic thresholds in vRealize Operations is probably being used here and I can see great value in the ability to adapt a whole application on the fly. On the other hand much can already be done with the publically available APIs and I can’t see how VMware would keep up with application revisions.  This was also on display last year with a slightly reduced scope so it’s obviously not a quick win!
  • http://www.amazon.co.uk/gp/product/B00I9PVKKC/ref=s9_simh_gw_p351_d0_i1?pf_rd_m=A3P5ROKL5A1OLE&pf_rd_s=center-2&pf_rd_r=0R9D23NVWYH3S7MY7NPV&pf_rd_t=101&pf_rd_p=455344027&pf_rd_i=468294High Performance Computing (Download the full PDF here). The last release of vSphere came with some specific features aimed at low latency applications but they come with considerable constraints to core features like vMotion. VMware aren’t resting on their laurels and are continuing to find ways of supporting low latency without constraints. High Performance Computing refers to Grid Computing and is often used in the sciences where crunching large numbers is commonplace. I had a good chat about the challenges and progress with Josh Simons from the HPC division.
    In these environments milliseconds count – if you want to understand why and enjoy a good read try Michael Lewis’s FlashBoys which tells the story behind high frequency trading! I read this on the flights to and from Barcelona so it won’t take too long but is recommended.

I also visited the VMware shop and found the selection of books now available to be very sobering! Compared to the early days of VMware there’s been an explosion in the complexity and breadth of topics you need to know. Two books caught my eye and are now on my Xmas wishlist – Cloud Computing by Thomas Erl  and cloud networking by Gary Lee.

Community Events

The blogger/community lounge (where I spent quite a bit of time) was nicely placed near the hands on labs and by the hang space. The vBrownBags (US agenda and sessions, Barcelona agenda), Engineers Unplugged (with guests Nick Howell and Gabriel Chapman), and VMworld TV crews were all in attendance doing their thing although sadly theCube wasn’t present as they only cover the larger US show. Lots of good content as always but sadly I missed this year’s vSoup VMworld podcast due to a clash with TechFieldDay Extra. This is the first time I’ve missed it since it started running in 2011. Instead they got John ‘the dude’ Troyer as a guest so it’s probably safe to assume I’ve lost my place for next year! Sad panda. 🙁

Yours truly debating EVO:RAIL

Taking my own advice I also went to the Meet an Expert sessions and had a couple of one on one sessions with VMware experts (Ninad Desai and Gurusimran Khalsa). This gave me the chance to put my question about the future of vCD directly to VMware staff although I had to go through quite a few people before I found someone who could give me a satisfactory answer (thanks Scott Harrison)! I’ve got a blogpost in the offing about this particular topic.

They often run one in the US but for the first time GestaltIT ran the TechFieldDay Extra event at the EMEA conference and I was invited as a delegate. I only saw two sponsors (X-IO and VMTurbo) as I could only attend one afternoon but both were interesting and as always there were good conversations both on and off camera with the other delegates (Andrea Mauro, Joep Piscaer, Arjan Timmerman, Marco Broekken, Martin Glassborrow, Nigel Poulton, Eric Ableson, Hans De Leenheer, & José Luis Gómez). It was fairly brief compared to a full event over a few days but still enjoyable and nice to meet a few new people who I’ve followed on twitter for a while. I was familiar with VMTurbo but X-IO were new to me – I’ll be posting my thoughts on both shortly. There was also a roundtable discussion on converged infrastructure which centred on EVO:RAIL – you can check out the videos by clicking on the logos below;

VMTurboLogoSm                             XIO-logo-wpcf_100x43

baby VMworld
For my 5 week old VCDX in the making…

I watched a few of the vBrownBag sessions – one on SSO by Frank Buechsel (@fbuechsel) and another good one from Gabriel Chapman on converged infra which was commendable for including actual customer numbers. I also caught, more by chance than design, the vExpert daily which is always fun if not overly informational. The vBrownbag sessions felt a bit unloved stuck to the side of the hangspace and I wonder if it wouldn’t be better within the Solutions Exchange, given that people are used to watching presentations there? I expect that wouldn’t work as it would have cost implications. I should also mention the portable whiteboard I got from vBrownBag – maybe it’s a novelty but at least potentially useful! The vBrownBag sessions were recorded and are all online via their YouTube page.

I didn’t party as hard this year as I’ve got a newborn at home so sleep (and a VMware vest!) was more of a priority – I skipped the vExpert/VCDX party, the Veeam party, and the official VMworld party. It also gave me a chance to write up notes, something I’d promised myself I’d do a better job of. I did kickstart the conference with the vRockstar party at the Hard Rock Cafe, which was great. I’ve got to know a lot of people over the last five years and it’s great to have a catch up over a drink and some tech chat. I spent much more time chatting about industry trends and canvassing opinion than previous years. I did make the PernixData party (great venue) and had a good chat with Ather Beg from Xtravirt and Chris Dearden, Ricky El-Qasem from Veeam and Canopy Cloud respectively.

UPDATE: 12th Nov

I also spent some time recording some sessions for VMware EMEA, talking people through what to expect at VMworld. It makes me cringe seeing myself on camera (that’s why I’m a blogger – I can write rather than talk) but you can watch it on the official VMware blog.