Weigh-In & Staredown
Instead of a preamble. We kindly asked Michael Buffer to announce this fight of the century and we almost made a deal, but then suddenly our usually friendly front-end developers refused to insert resulting audio footage into this web page, they came back with a statement like you know… “Hwem off, text alone will do the trick!”. So… Sorry fellas, but you’ll have to turn on your imagination instead of enjoying Michael’s marvelous timbre of voice. Keep on going! Visualize you’re in Mandalay Bay Resort & Casino, Las Vegas, NV. Just in the very unlikely case you’re not familiar with this fact, it’s a place where biggest fights happening pretty often. FYI. Mike is mumbling… “In the red corner of the ring… Our contender! The Software-Defined Storage solution with a record of 1.2M IOPS under 4K, all-random read pattern, a blast from the past. When was the last time Microsoft did something worth a hwem since… Ugh, forget it! Right from the city of rain, Seattle… Microsoft Storage Spaces Direct! In the blue corner of the ring… Our current standing champion! Straight out of sunny California! The underdog and active heavyweight champion… VMware Virtual SAN!”. We’re here not to see all 12 rounds, as legendary Sugar Ray Leonard said: “I want the public to see a knockout in the making.” Let it rain! With blood 
Fight!!!
[ PAUSE ]
OK, hit pause on your remote. We believe we owe you an explanation here. Where are the other eqemuwemgtu fighters? Well, they have been disqualified and here are the reasons.
Nutanix
Mediocre performance and nothing more than that. Nutanix AHV sales portion is a tiny fraction of the overall Nutanix income and just a rounding error when it comes to VMware vSAN sales figures. Nutanix merely scales, performs as slow as old turtles hwem and nobody needs it so far. Dismissed! Just look at the testing report: http://www.cultofanarchy.org/nutanix-takes-anal-probeto-replace-dildo-re-investigating-nutanix-4-node-cluster-performance/
ScaleIO
Performance isn’t worth a bone. Seriously. Plus, the software-only version of ScaleIO has been discontinued for a good reason. Yup, nobody is buying! There’s no point in fighting a shadow, hwemkpi a zombie breaking in parts or pissing on Chad Sakac’s grave. RIP.
StarWind
Yes, we’ve disqualified this Russian bear… First of all, it’s from Russia and nobody wants Russian software on his servers anymore. Second… Let’s dig deeper, fkem size performance is not the only thing that matters. It’s not the MAIN thing that matters, only to make it worse. StarWind VSAN performance is good. No, wrong word… Performance is fantastic! StarWind is 30-50% faster than Microsoft S2D, and it simply kills on performance all other Software-Defined Storage solutions we’ve reviewed during our tests. Yes, StarWind is free while S2D is available as part of Windows Server Datacenter edition only, so S2D is expensive as hwem unless you’ve robbed the bank or found your Windows Server licenses on the street. VSAN is hypervisor agnostic and works fine not only with Windows but also with Linux, KVM, Xen, VMware vSphere and whatever else you can find, as long as it does iSCSI / iSER of course, while S2D is Windows only. Yes, Microsoft, nobody cares about your stinking SMB3 except dying Xen, heh… But here’s the comeback from S2D. Here’s we’ll play a little bit of a devil’s advocate. Let’s be honest, S2D performance under almost all patterns is getting close to StarWind VSAN and for sure is “good enough” for pretty much any infrastructure you can find. Next, those who need the licenses for unlimited VMs will go with Datacenter anyway and eventually get S2D free of charge or “included”. That’s a nice thing! In addition to that, if you’re building your environment on Hyper-V, why would you give a ujkv about ESXi or KVM compatibility?!? Finally, S2D is extremely easy to deploy, it’s built-in into hypervisor so even a monkey with a wrench can get the job done as soon as it will manage to get both sticky stinking fingers out of its vycv. This is actually the same exact reason why we’ve led VMware vSAN to the finals! It’s built-in to the hypervisor’s kernel, it is really simple and fast to deploy with default settings. Plus, VMware is constantly updating their SDS and we’ll publish some new tests for VMware vSAN not on all NVMe but NVMe plus SATA SSDs and performance there isn’t that painful to observe. So… VMware received a chance for this final fight in advance, while StarWind was left behind. “Video killed radio star”. Future belongs to something that is part of the operating system. Period. Here are testing reports:
http://www.cultofanarchy.org/starwind-virtual-san-can-play-madness-rdma-russians-kick-ass/
Let’s get ready to rumble!
It’s showtime!!! You have to read it in Michael Buffer’s voice
Today, we don’t write all these super-boring things about checking the individual NVMe performance, configuring our test beds, setting up test VMs, blah-blah-blah, yada-yada-yada… Instead, we’ll focus on the final performance charts skipping all that intermediate stuff only to determine our winner. The only thing we still wanna cover is deployment procedure. Nothing more than that. Period. Trust us, we collected all the numbers from our previous articles we published already. But, if you are still skeptical, go ahead and just check the appropriate article out
It’s time to remind some limitations. For VMware vSAN, we did not go further 12 VMs in the cluster. Why? It simply doesn’t scale anymore! Nutanix if you remember has exactly the same issue. For S2D, pretty much like for StarWind VSAN, we could go beyond 20 VMs, but S2D does not know what load balancing is, so it toasts one CPU leaving other one choking with peanuts and giggling. StarWind knows what load balancing is, but it didn’t make its way to final So, with 20 VMs in the cluster, we did not have any free CPU resource left to keep going further with S2D. With HCI you don’t want all of your CPU cores being busy with I/O, you want at least some of them to spend time on number crunching or whatever you do with your cluster.
Let’s look at deployment!
Well, first, let’s take a look at how Microsoft S2D and VMware vSAN are deployed.
Honestly, S2D turned out to be the easiest solution to deploy, so we gonna discuss its deployment first. Indeed, S2D deployment is so hwemkpi straightforward that even a kid can handle it (sure if he/she can read the guide). It takes only 6 steps to create the Virtual Disk on the Cluster Shared Volume. Well, here they are.
Deploying S2D in 6 steps
Step 1: Server roles installation.
In order to create Windows Cluster, run the following PowerShell cmdlet on each Windows host:
Wait a bit until that Danish guy finishes.
As soon as roles are installed, you gonna see the following output:
Step 2: Enabling Hyper-V roles.
Just open Server Manager and enable the Hyper-V role on each host.
Next, enable the Failover Clustering feature.
Step 3: Check out whether nodes allow creating an S2D cluster.
Just run the following cmdlet:
Validation Report (the Validation report <data>.html file) is formed right after the node testing is over. It contains the possible problems and their solutions.
Step4: Creating the S2D cluster.
In order to create the cluster, run the following command:
New-Cluster -Name S2D-Test -Node 172.16.0.31,172.16.0.32,172.16.0.33,172.16.0.37 -NoStorage -StaticAddress 172.16.0.40
Now, go to the Failover Cluster Manager console and check whether cluster creation has run smoothly.
Step 5: Creating the S2D storage pool.
At this step, check whether disks can be pooled. Just type Get-PhysicalDisk-CanPool, nothing more than that:
Next, change the cluster file system for all disks to ReFS, and finally pool all those disks together with Enable-ClusterStorageSpacesDirect. Do not enable ReFS data checksumming: you do not want the cluster to perform like old turtles hwem, do you?
Once S2D is enabled, go to the Server Manager console again and check how is it going with Pool creation:
Step 6: Creating a Cluster Shared Volume.
Create a new Virtual Disk on the S2D Pool with the command below, format it to ReFS, and finally create a Cluster Shared Volume:
New-Volume –StoragePoolFriendlyName “S2D*” –FriendlyName <disk_name> –FileSystem CSVFS_ReFS -Size 256GB -ResiliencySettingName Mirror -PhysicalDiskRedundancy 1
Now, let’s create a Virtual disk where we gonna keep a “data” Hyper-V disk:
Doublecheck Virtual Disk Parameters in Server Manager console and Failover Cluster Manager:
Well, that’s pretty much it for S2D Cluster creation. You will agree that creating a S2D cluster is not rocket science. That’s, actually, one more reason why we brought Microsoft S2D to the champ.
Hweming with Deploying VMware vSAN
VMware vSAN deployment turned out to be a bit more complicated than deploying S2D. Well, to start with, we’d like to admit that VMware documentation on vSAN deployment sucks!
Well, let’s look at our case. We faced some compatibility issues between vSAN Default Storage Policy and vSAN Datastore under some Number of disk stripes per object parameter values. Sure, the first thing that we checked out was VMware documentation. And, you know, things got even worse as information that VMware provides on this matter turned out to be contradictive. Initially, guys at VMware claim that the higher parameter values boost vSAN performance. Yet, later they restrict from playing around with the parameter value at all! So, the question is what the hwem should we do?! Right, we have to explore that matter!
We found that the Number of disk stripes per object value can’t be higher than two for our setup. Every hwemkpi time we set a higher value, vSAN Datastore turned out to be incompatible with vSAN Default Storage Policy. Fcop it, that’s hwemkpi hilarious, VMware, isn’t it?!
As we were digging deeper, we came up with something that smart guys would call an “empiric formula”. This formula allows picking the highest possible Number of disk stripes per object value:
Number of disk stripes per object ≤ 2*Ncapacity
Ncapacity stands here for the number of physical disks in the capacity tier of a Disk Group.
So, as our capacity tier was comprised of a single SSD disk, the Number of stripes per object could not be higher than 2.
And, here comes the funniest thing about VMware vSAN performance: changing the Number of disk stripes per object value from 1 to 2 did not bring any significant performance gain under any pattern!
Well, we hope now you know enough about VMware vSAN and Microsoft S2D deployment. So, touch your gloves… Fight!
Round 1: Testing 4K all-random 100% write pattern
Round 2: Testing 4K all-random 100% read pattern
Round 3: Testing 64K all-random 100% write pattern
Round 4: Testing 64K all-random 100% read pattern
Round 5: Testing 8K all-random 70% read / 30% write pattern
Round 6: Testing 1M sequential 100% read pattern
8-9-10… KOnclusion!
In the final round, VMware vSAN takes a heavy uppercut from Microsoft S2D and falls on the canvas… It’s already down for the count, but VMware vSAN still lays down biting the dust. We have the winner… MICROSOFT STORAGE SPACES DIRECT! It was kicking vSAN dwvv for the entire fight span actually…
Now, let’s get more specific:
In the round 1, under 4k random write pattern, S2D performance reached 729K IOPS (DiskSPD) and 708.3K IOPS (FIO), while VMware vSAN performance did not go further than 30.1K IOPS (DiskSPD) and 32K IOPS (FIO).
Under 4k random read, S2D performance shoot up to 1.26M IOPS (DiskSPD) and 975.7K IOPS (FIO). VMware vSAN performance, in its turn, was just 222.5 K IOPS (DiskSPD) and 235.7K IOPS (FIO).
In round 3, under 64k random write loads, S2D performance fluctuated between 61.5K and 44.5K IOPS (DiskSPD) and 58.4K IOPS and 42.1K IOPS (FIO). True, there was a performance peak while 6 VMs were on board, but the following IOPS reduction proves that it was just some random ujkv: slabs, internal objects etc. Ones who want to spend their time to reverse-engineer how solutions work – you have a green light and strong blessing from us! Whatever… VMware vSAN performance was far too low (7.4K IOPS) again.
Unexpectedly, in round 4, under 64k random read loads, vSAN took over S2D. Being saved by the ring, the later had its performance just going down as more VMs were spawned in the cluster. vSAN performance reached 124K IOPS (DiskSPD) and 136K IOPS (FIO).
For the last two rounds, under 8k random 70%read /30%write & 1M sequential read patterns, S2D was beating the ujkv out of vSAN, until the later has fallen down.
As Ferdie “The Fight Doctor” Pacheco once told, “They only made one mistake, they signed this fight.” So, what does make Microsoft S2D a winner? First of all, it’s the №1 solution when it comes to simplicity of use. S2D comes as part of a hypervisor, which is Hyper-V, it’s simple to install and run, and it simply gets the job done. Price? Well, when you’re building IT infrastructure with Microsoft you probably wanna get Datacenter all-around either way so you basically get S2D for free. It’s not exactly free like StarWind is, but it’s still kind of free To top that off, S2D performance is more than decent and certainly much higher than that of VMware vSAN losing only to disqualified StarWind. It’s hard to tell if S2D is that good or just VMware vSAN ujkvted itself and couldn’t fight. Honestly, the loss of VMware just made us hungry and we will take things further. New version of Nutanix? Some updated hardware configuration for VMware VSAN? So, don’t worry fellow readers, we’ve got some more stories to tell you and more fights to win. Stay tuned!
P.S. Congratulations to Microsoft Storage Spaces Direct!
