Getting Moore Out of Your Cloud
As many of you in the technology industry are aware, in 1965, Dr. Gordon Moore, co-founder of Intel, observed that the number of transistors producible within a silicon wafer could economically double every year. Although the doubling rate has been revised several times since then, the prediction has served as a target and benchmark for semiconductor technology growth.
Moore’s law has fueled exponential computing advancements over the last several decades. For example, we have seen microprocessors with a few thousand transistors and kilobytes of memory transform into multi-core server chips with billions of transistors, gigahertz clock rates, and equally impressive storage and networking capabilities. Today, it is the availability of these servers with their exponentially improving roadmap over time that underpins hyperscale and cloud buildouts. No exponential, however, lasts forever, and understanding the future limitations of Moore’s law may be useful in understanding the challenges and opportunities that lie ahead for continued cloud growth.
Today, mounting evidence suggests that Moore’s law is running out of steam. The once doubling pace has slowed to about every three years and most analysts project an end to its progress in the middle of next decade due to the fundamental limits of semiconductor device physics. Clearly this signals a change in business as usual.
In the virtualized world of cloud software infrastructure, opportunities for software improvement abound, but not all areas are likely to yield the same returns
The challenge that a slowdown of Moore’s law poses to cloud and hyperscale growth should not be underestimated. Without continued server improvements, larger numbers of servers must be deployed to keep up with the increasing demand for cloud infrastructure. For instance, if Moore’s law continues but doubling at a rate of once every four years instead of today’s once every three years, an increasing gap of 6% more equipment per year is required to make up this difference. This compounding 6% not only impacts capital expenditures for equipment and facility space, but also translates into increased operational expenses due to cooling costs, maintenance, and power consumption. Of course, if Moore’s law ends completely, the results are even more dramatic - the server gap will increase by 25% each year. Therefore, aggressive measures will be required to fill this gap if cloud growth is to continue unabated.
Fortunately there are some options available to address the server gap caused by a flagging Moore’s law. The first of these options is simply investing in software efficiency improvements. Ironically, this option may have been devalued in the past as a result of the success of Moore’s law. After all, why invest in software improvements if the next generation processor will make even the most sluggish code appear snappy? If processor improvements become scarce, a renewed focus on software efficiency becomes reasonable.
In the virtualized world of cloud software infrastructure, opportunities for software improvement abound, but not all areas are likely to yield the same returns. Hypervisors, for instance, have undergone intense industry scrutiny in every stage of their development are therefore unlikely to be a source of significant continued improvement. On the other hand, application code is likely ripe for investigation. As an example, when working in non-virtualized environments, my team regularly identified changes that resulted in code speedups of 10 to 100 times when compared to non-optimized code. Improvements such as these, if consistently identified in cloud applications, would close the Moore’s law gap for a decade or more.
A second potential opportunity to close the Moore’s law gap is available by focusing on processing architecture. Today’s servers have evolved to support a variety of workloads but are not tuned for any in specific. In the past, however, application specific processors (digital signal processor, packet processor, etc.) were more prevalent and offered advantages in both performance and power consumption compared with general purpose hardware. Because of the steady pace of Moore’s law fueled improvements to servers, their larger skills base, and more mature tool chains, general purpose processors won the day. However, with the decline in Moore’s law, the dynamics shift favoring a mix of application specific processors and general purpose processors to close the gap.
A good example of how application specific processors and high-volume servers work together can be seen in today’s deployments of GPGPU (general purpose graphics processing unit) processing solutions. GPGPUs employ large numbers of processing cores and massive amounts of parallelism to provide improved performance for a variety of graphics, video processing, and simulation applications. As an example, the Artesyn SharpStreamer™ GPGPU product, when combined with industry-standard servers, was able to provide 24 times better processing capacity compared to using standard servers alone. With numbers like these, targeted use of other application specific processors could help close the Moore’s law gap for more than ten years.
Of course, there are always challenges with any new solution approach and these two are no exception. Most notable in both of these cases is some loss of flexibility. Extreme code tuning to fit specific processor architectures will result in the highest performance gains, but will also result in a rigid cloud in which software is tightly coupled to hardware and code is not easily migrated. Likewise, hardware acceleration through application specific processors tunes the underlying cloud hardware to excel at certain workloads at the expense of others. Since flexibility is a hallmark of today’s cloud infrastructure, special care may need to be taken in future cloud deployments to preserve this capability.
Finally, it should also be noted that neither software tuning nor hardware acceleration is the ultimate solution. Neither can provide continued improvements indefinitely and at some point diminishing returns will set in. If however, these measures along with others can buy a couple of decades of relief, it is highly likely that a silicon replacement technology will be available that can pick up where Moore’s law left off. I’ll let you speculate what the replacement technology might be, but one thing is sure - with the phenomenal success of Moore’s law, it’s not likely to stay down for long.