Friday, 06 April 2018 17:51

FPGAs play an interesting role in Azure


You might be surprised to learn the part FPGAs are playing in Azure's hardware.

Field-programmable gate arrays (FPGAs) can be seen as an intermediate step between conventional CPUs and ASICs (application specific integrated circuits).

To quote Wikipedia, "FPGAs contain an array of programmable logic blocks, and a hierarchy of reconfigurable interconnects that allow the blocks to be 'wired together'" to perform a required function.

While an ASIC is likely to be faster, its functionality is fixed, whereas an FPGA can be reprogrammed if desired. That could be to fix bugs, add new functions or reimplement existing functions in a more efficient way.

FPGAs can provide higher performance per watt than CPUs or GPUs for certain functions.

So Microsoft uses FPGAs to implement SDN (software defined networking) for Azure.

According to Microsoft Azure CTO Mark Russinovich, processing 40Gbps of traffic in and out of a server would occupy eight full Xeon cores, which are then no longer available to run customers' virtual machines, as well as introducing latency into the network flows.

Microsoft Azure CTO Mark Russinovich

"Microsoft had already started to develop some expertise in FPGAs and when we looked at it we thought maybe these could solve the problem," he said.

ASICs would give the highest performance, lowest cost and lowest power consumption. "But the challenge we had when we considered ASICs is our software defined network continues to evolve and get new features all the time. If we locked ourselves into an ASIC that takes 18 months to two years to come out with, by then our software defined networking requirements would have changed and we would be stuck with something that was two years old and we would have to try to use that one for at least two years to make our investment worthwhile."

FPGAs, on the other hand, can be reprogrammed as required. "About once a month we update our stacks on these FPGAs across the fleet to fix bugs, improve performance and add new features.," he said.

Using FPGAs can reduce latency by an order of magnitude, for example from 500 microseconds to 50 microseconds or lower, he said, "with zero usage of the [CPU] cores.

This technology is able to support 50 and 100Gbps networking, Russinovich suggested.

"This approach lets us adapt more easily, basically futureproofing us, while driving world-class performance," he told iTWire.

FPGAs are also being put to work in Azure's AI services.

Training a deep learning model is very computationally intensive, and so GPUs are almost universally used for that task.

But when it comes to inferencing (eg, getting a trained model to identify the object in a photograph, the Mandarin equivalent of an English sentence, or the sentiment expressed by a piece of text), it can be hard to keep a GPU busy.

Interactive situations call for instant responses. "You don't want to sit there and wait for a batch of those queries to build up before you submit it to the hardware", but that's what is needed to make the most use of a GPU, he said.

"You just want to send in the queries and get them right back. GPUs are just not able to do realtime AI today. But we realised again with our investments in FPGAs we could put in deep learning models and get realtime [responses] back. That's a project called Brainwave."

Russinovich showed a demo that delivered inferencing queries to a GPU in progressively smaller batches, and as the batch size dropped, so did the number of operations performed per second, as a growing proportion of the GPU's resources sat idle.

But with an FPGA-based inferencing implementation, the number of operations remained steady at around 40 teraflops whether queries were delivered in batches of 256 or individually.

"We're pushing the envelope for machine learning infrastructure in Azure," he said, not just by offering the latest and greatest Nvidia GPUs but also by investing in alternative technologies such as FPGAs.

The Brainwave API would soon be available to Azure customers, Russinovich said.


As part of our Lead Machine Methodology we will help you get more leads, more customers and more business. Let us help you develop your digital marketing campaign

Digital Marketing is ideal in these tough times and it can replace face to face marketing with person to person marketing via the phone conference calls and webinars

Significant opportunity pipelines can be developed and continually topped up with the help of Digital Marketing so that deals can be made and deals can be closed

- Newsletter adverts in dynamic GIF slideshow formats

- News site adverts from small to large sizes also as dynamic GIF slideshow formats

- Guest Editorial - get your message out there and put your CEO in the spotlight

- Promotional News and Content - displayed on the homepage and all pages

- Leverage our proven event promotion methodology - The Lead Machine gets you leads

Contact Andrew our digital campaign designer on 0412 390 000 or via email



Security requirements such as confidentiality, integrity and authentication have become mandatory in most industries.

Data encryption methods previously used only by military and intelligence services have become common practice in all data transfer networks across all platforms, in all industries where information is sensitive and vital (financial and government institutions, critical infrastructure, data centres, and service providers).

Get the full details on Layer-1 encryption solutions straight from PacketLight’s optical networks experts.

This white paper titled, “When 1% of the Light Equals 100% of the Information” is a must read for anyone within the fiber optics, cybersecurity or related industry sectors.

To access click Download here.


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments