Yes, I know some of you will say SAN (FC, iSCSI, etc.) is here to stay, and in some enterprises storage pros will explain why it’s faster and better and why the high TCO and complexity are inevitable. But let’s examine it in light of recent developments, or simply for the sheer fact that Google, Amazon, Microsoft and other large data center operators don’t use SAN.
First, let’s start with a quick overview – SAN stands for Storage Area Networks, and usually means some sort of SCSI mapping to network frames (FC FCP and iSCSI). Virtual disks also known as LUNs (Logical Unit Numbers) are mapped to a remote client over the network and the client “sees” them as local disk resources.
FC (Fiber Channel) is still the dominant SAN fabric. It requires a dedicated network fabric, adapters, and is managed by a separate and specialized team in the organizations with their unique disciplines and skills.
By nature SCSI or any other block storage is not a shared resource, in order to have multiple applications access the same LUNs, you would need a shared cluster file system with distributed locking – not the most scalable solution, and a very complicated or proprietary one. So in most cases SAN will be used as a replacement to local disks, or as shared storage for databases which maintain their proprietary distributed volume manager (e.g. Oracle ASM).
When people need shared storage which can be accessed by any type of client, whether it’s an enterprise application, a desktop, or a mobile device, they use file services, and in cloud scale use object storage. So why not use forms of shared storage like file and object in all application use cases? After all with the trend towards cloud and big data it’s all about sharing and reusing data and not about keeping it in isolated silos.
Now the critics will say “you don’t know what you are talking about – SAN is much faster, and it’s the right thing for virtualization, databases, etc.”, so let’s examine some of those myths one by one.
Myth #1: SAN is best for Virtualization and VMware
All the recent news in Virtualization point to converged infrastructure (CI) with solutions like VMware EVO RAIL, EMC ScaleIO and Nutanix, or at larger scale the use of shared file/object solutions like Ceph, Microsoft SMB 3.0, or Amazon/Google proprietary scale-out storage. Those solutions are proven to scale, are faster, simpler and have lower TCO than using FC or iSCSI SAN. They treat virtual disk volumes as objects which can grow/shrink, upload/download, snap, cache, have unique policies and more, none of those features are part of the 30+ year old SCSI standard. There is no need for layers of SCSI mappings (Map hardware LUNs to protocol LUNs, map those to hypervisor LUNs, and again map it to VM LUNs) or proprietary SCSI extensions like VAAI in which you lose context and ability to manage the VM resources, instead we need object APIs to create and manage virtual disk objects.
Myth #2: SAN is a must for Databases
In the legacy datacenter SAN arrays were primarily used for databases, and allowed maximum performance, creating this false notion (that was true 10-15 years ago …). But looking at modern examples and BigData, the faster and most scalable solutions are based on smart file or object storage.
Just a few notable examples:
- Oracle Exadata is using an InfiniBand backbone with special object like protocol between storage nodes and DB nodes, which provides query offload semantics, and allows a magnitudes faster performance compared to any SAN technologies.
- Microsoft uses SMB 3.0 file protocol with RDMA based hardware to show off its latest MS SQL benchmarks. SMB 3.0 provides way more IOPs and lower latency at the application level compared to common SAN infrastructure.
- Amazon Aurora provides up to five times better performance than MySQL and is built on top of Amazon scale-out and object storage framework.
- When examining Hadoop, BigData, NoSQL and NewSQL solutions, they are all built on top of new scale-out file or object storage paradigms. None of them mandate SAN hardware.
So the right solution for databases and BigData is to build scalable high-performance file or object storage, which can optimize the path from the application to the various storage media options.
Myth #3: When using large file systems use SAN as a backend?
Yes, many use SAN as a way to aggregate lots of storage behind few computation boxes, but they require complex synchronization and lock management between the computation boxes, something which can be greatly simplified when the storage is exposing individual objects to the applications or file systems, and those objects can be monitored, updated, versioned, or locked at fine granularity. Just like in the case of CephFS, Scality Ring, or Seagate Kinetic hardware, we need software defined “Object Area Networks” based on commodity hardware to enable highly scalable and cost effective capacity growth in the age of BigData.
And what is my conclusion?
Scale-out, software defined object storage is the right way to build storage in the new age, and those objects need to map to VM disks, files, or consumed directly by databases or new age applications. Object also opens the door to new and more efficient APIs, if my application needs to get data, why do we need to do lookup(), open(), read(), close(), and not a simple get() with lower overall latency and overhead? Or why won’t the storage filter the results upfront like in Oracle Exadata or Amazon S3 do?
With new flash and disk technologies object and key/value APIs are a better match than SCSI mappings, since new disks or flash devices manage and allocate their own resources to provide FTL, journaling, compression, avoid faults, etc. This is yet another reason to build object solutions which can map directly to new object based physical media, instead of over a redundant layering of SCSI/ATA protocol on top of the object media.
Yes, unfortunately the current open source object storage solutions are usually slow and “eventually” consistent, quite limiting its use to backup and archiving. And the good solutions are kept proprietary by the cloud or database or CI vendors. But it’s time to change the paradigm and form industry standards for faster and more general purpose object storage APIs over traditional networks, and create standards for object storage management, just like we have them on SCSI or file based networks.
My take is that IT shops or storage vendors which focus on complex and high TCO solutions like SAN will face a challenge in the long run. They would have to compete with the large cloud infrastructure providers on the business, and those already moved to the new and much more efficient paradigms, they can build and maintain storage at larger scale, keep it at fraction of the cost, and backup/replicate it across the globe.