PernixCloud Data: FVP Latency Compared To SAN Latency

We have been exposing data from our PernixCloud in articles such as A Portal Into The Planet's Virtualized Data Centers , Insights Into CPU And Memory Configuration Of ESXi Hosts and Insights Into VM Density .We collect lots of data of the virtual datacenters around the world to understand the architectures companies operate, but we also use the data to understand the performance of our own products. One of the key metrics that we are interested in is how much latency of workloads we improve when accelerate with FVP. Satyam asked Woon Jung and Sriram Sankaran to dive into the dataset and reveal how much latency is reduced. Below are the comments of Satyam when presenting the data:

The Dataset
Satyam: We observed ~ 91000 VMs across multiple days, because a VM IO personality changes at different times of the day and different days of the week. Each colored segment of the donut charts below represent the percentage of VMs that show x-y% of latency reduction compared to the customer's SAN latency. For example, the dark(est) blue segment of the top left inner donut chart says that 5% of total write through VMs running on VMFS around the world have seen a 0 to 10% latency reduction with FVP.

Please read this article to get familiar with FVP and the write policies:

In total we collected over 1382 VM-years of datapoints. The data is displayed using the standard consumption metric VM-Hour, similar to various cloud models such as EC-2.

Latency reduction for write through VMs on VMFS - Read (inner donut) and write (outer donut)

Latency reduction for write through VMs on NFS - Read (inner donut) and write (outer donut)

Latency reduction for write back VMs on VMFS - Read (inner donut) and write (outer donut)

Latency reduction for writeback VMs on NFS - Read (inner donut) and write (outer donut)

Satyam: Ready for the kicker? Well, many of them, but here is a sample and the rest is left as an exercise for the reader:

  • 35% of total write through VMFS VMs running around the world have seen 100s of percentage points of latency reduction -- i.e. VM observed latency is half or less than half the SAN latency.
  • Running VMs in write through indeed helps write performance as well because the SAN is less loaded. We see 14% of VMFS write through VMs across the world see WRITE latency decrease in the 80-90% range. That is HUUUUUUUGE.
  • In the top-right NFS chart, one sees that NFS VMs see a much bigger decrease in latency with FVP, because NFS sucks (performance-wise).
  • That brings us to write back, half of all VMFS VMs running in write back see 100s of percentage point drop in latency when running in write back. Ergo write back is the bees knees.
  • In general, you will see that most VMs fall in the 50%+ latency reduction bucket, whether it is reads or writes, write through or write back, VMFS or NFS. Consistent bad-assery

Phenomenal data as you can see, now the interesting part is the impact on SAN write latency reduction via write-trough. This is real data and think about FVP with RAM acceleration (DFTM) in front of an All-Flash Array. Recently Ramboll, a Danish engineering company with a global presence gave insight into their reason why they used FVP with DFTM in front of a Pure All-Flash Array. You can listen to the recording here: Maximize Your All Flash Array Investment With Analytics

Chethan Kumar, our storage performance engineer point out the fact that the reported latency of the SAN is lower than if no FVP is active. Due to the amount of bandwidth saved, the overall load on the storage area network and the storage controller is reduced. If all the I/O's of the 90.000+ virtual machines were issued, the overall latency would be much higher. In that regards the improvement could be much more than what these donuts are showing. The real donuts should mostly be purple with few other patches towards the top.

Don't underestimate the amount of bandwidth virtual machines with write-through would provide. This week I visited a customer who are accelerating 32 virtual machines running databases and they are saving over 145 TB of bandwidth a day. Think about the nominal bandwidth reduction of the storage area network and the impact it has on the caching of the Storage array.


To show we are very confident in the benefits that FVP can provide we are running the contest "Fastest SAN alive". Take up the challenge and see that FVP can provide Up to 10x Faster VM Performance. Details can be found here.

fastest SAN alive

Frank Denneman

Follow me on Twitter, visit the facebook fanpage of, or add me to your Linkedin network