There Actually IS a Free Lunch
with WekaIO’s Zero-Overhead Encryption!
Andy Watson. December 20, 2019
Andy Watson, Chief Technology Officer at WekaIO, shares his thoughts about WekaIO’s encryption capability having zero performance penalty impact in this blog titled “There Actually IS a Free Lunch”.
When people find out about WekaIO’s end-to-end data-encryption capability (see my October 2019 blog, “Going All The Way (With WekaFS™ Encryption”), they almost always ask right away whether there is any associated performance impact. “What is the overhead?”
One answer we can provide is, “It depends.” But that might be misinterpreted as evasion. Another response we sometimes offer is a range: “15 to 30 percent.” But how will your mileage vary and which factors affect it?
Recently, we have been seeing actual results at customer sites that several of our architects had predicted but no one at WekaIO was willing to shout out from the rooftops until there was proof in hand: zero performance impact!
This is worth discussing — because it’s exciting and, frankly, because it’s more than a little hard to believe.
“There ain’t no such thing as a free lunch.”
— Robert Heinlein
The Moon is a Harsh Mistress (1966)
Encrypting/decrypting data involves additional computational work, and that cannot possibly occur in zero elapsed wallclock or CPU time. If you’re skeptical about this encryption lunch not being free, obviously you aren’t wrong. However, there’s another perspective which can account for the experience some of our customers have reported — that they have enabled encryption and simply have not observed any loss in performance.
Let’s put it under the microscope.
If we focus on a single small-block I/O event, then it’s safe to assume that it will definitely and reproducibly take longer to complete. This is an unavoidable effect: in addition to completing all associated metadata transactions, the Read() or Write() operation will also incur encryption or decryption calculations.
However, most application workloads entail a pipeline of such events to be handled by the file system in series. The additional latency associated with each event can take place while other separate events are also progressing through the execution of their own instruction stacks. The cumulative result is assessed as “throughput” while the response time associated with each individual event will be measured as “latency”. The latencies per individual operation can overlap, and at the application level, the effective performance becomes more a function of the throughput than the response time. That is how it is possible for some of our customers to enable encryption while observing zero performance impact.
Let’s test this thought experiment with some real numbers. Suppose the additional latency added to each small-I/O operation is about 0.15 milliseconds. (Based on our testing, that would be conservative: it’s 30% of the typical 0.5 millisecond average response time.)
If there were only one single event at the application-relevant level, then the observable performance impact would be 30%. Very simple.
If there were two of these I/O operations in the pipeline relevant to the application, then the latency would overlap and the 0.15 milliseconds would be amortized over the whole transaction. That would cut the overhead to 15%.
More realistically, most applications generate tens of thousands or even millions of I/O operations. Let’s look at a pipeline of 10,000. In that case, the effective overhead would be reduced to 0.003% — and most customers would consider this “zero impact” because it would be effectively unmeasurable.
At the macroscopic level of a real-world production environment, there are usually dozens or hundreds of application instances. As we approach an infinite number of I/O events, the amortized latency will indeed approach zero at the limit.
“There’s no easy way to be free.”
— Pete Townshend
“Slip Kid” (1976)
Some of you could hardly focus on the previous section of this blog because you’ve been down this road before and you know where it goes.
Yes, there are circumstances where the overhead would be noticeable. Under heavily loaded conditions, there won’t always be CPU cycles freely available to perform additional encryption-related work. In such situations, the latency associated with waiting for CPU resources would be further compounded as the queues of waiting tasks grow longer. Instead of providing an aggregated flow over which the latency could be amortized, those pipelines of I/O events would make the situation worse.
For this reason, when WekaIO systems are being configured to also enable optional encryption services, we sometimes recommend a modest increase in hardware resources.
In conclusion, we can see that the encryption lunch was not literally free. But if you are already paying for it in the form of available CPU resources, then there is no additional cost and the encryption lunch is effectively free. WekaFS is designed to efficiently harness those otherwise-idle CPU cycles. The result is zero-overhead end-to-end encryption, protecting your data both in-flight across the network and at-rest.