VMware had a big party this week, many new product introductions, mostly storage focused, and some buzz around DevOps, Micro-Services, and Cloud-Native applications. But beyond the buzz and vision announcements seems like VMware and most of its partners are still focused on providing incremental improvements for today’s IT (a.k.a “Private Cloud”), vs cater to DevOps or provide Enterprises a bridge to public cloud economics or micro-servicesarchitecture.
Some background to Micro-Services and Cloud-Native
Cloud-Native is a new elastic services model targeting public cloud deployment, with scalability, agility, fault-resiliency, and continuous integration in mind. To address that new tools and services were developed like Docker containers, Kubernetes, elastic services for log, SQL/NoSQL, messaging, etc.
The new architecture revolve around micro-services, i.e. tiny stateless and autonomous application fragments doing one task well, many similar micro-service instances can be deployed (using Docker), grow and shrink to form a services, and are part of a bigger and evolving application.
According to the 12 Factor methodology service instances must not persist any configuration, logs, data, or even dirty caches. For that we have scale-out persistency layers to handle log streams, objects, and files (e.g. MongoDB, Elastic Search, Cassandra, ..), isolating developers from the very complex art of data indexing, consistency, integrity, atomicity etc. Can see a good overview of Netflix architecture as a model.
It is quite different than the traditional monolithic/scale-up Enterprise apps and current IT infrastructure approach, where apps have lots of configuration, data and state (Stored on “Virtual Disks”), failures or upgrade can lead to service disruption, dirty caches lead to inconsistencies, databases don’t scale, and SAN/NAS storage prevail.
Going back to VMworld
If public cloud is the major threat to enterprise IT vendors, and micro-services are the future, why don’t we see more of that coming from VMware and their partners, and we mostly see repainted legacy IT tools. Luckily VMware have adopted Docker this year, after they ignored/bashed it last year, but we need a change in mindset.
Some examples I noticed:
- VMware main storage announcement was around VSAN (a distributed hyper-converged block storage), sounds cool, but did you know Docker only works with files ?
- So many storage vendors brag about their Deduplication support, did anyone notice Docker uses a layered file system which eliminate duplicate files by design (and has tiny images)?
- New tools for simplifying block storage snapshots and backups, guys, just a reminder, in micro-services the image is static, what we need to back-up/manage is the data.
- Many tools help make the IT and VM admin life simpler, but what about the dev-ops guys? And where is the self-service model where users can manage and provision the resources/data (not infrastructure) just like they can in Amazon or Azure.
We saw a huge focus on hyper-converged infrastructure, where the data is kept in the individual cluster, in most cases can be accessed by a single VM at a time (vDisks), fixed ratio of CPUs/Disks, and replication across nodes. This is a big win for legacy IT, no more FC SAN hassle, VM admin can do it all, but this doesn’t align with Cloud-Native, which have the opposite requirements:
- No persistent data in the micro-service image, and it’s quite small (i.e. why replicate, distribute, dedup or snapshot)
- The persistent data is stored in things like Object store, MongoDB, Cassandra, ELK, .. which have built-in replication and distribution.
- Data is accessed via file or object APIs, not as a vDisk
- Unstructured data tends to grow constantly and even exponentially, so co-locating CPUs with Disks is not the best economics.
- Data must be shared by multiple clusters, apps, and even mobile devices, not be in silos.
Yes, VMware and co need to support the existing customers which haven’t yet adopted the new model, but if Enterprise IT vendors want to stop the threat from the public cloud they need to take more aggressive steps in adopting public cloud mind set.
See more in the next post on how to create Enterprise infrastructure for Cloud-Native apps.