Friday, 06 April 2018 17:51

FPGAs play an interesting role in Azure


You might be surprised to learn the part FPGAs are playing in Azure's hardware.

Field-programmable gate arrays (FPGAs) can be seen as an intermediate step between conventional CPUs and ASICs (application specific integrated circuits).

To quote Wikipedia, "FPGAs contain an array of programmable logic blocks, and a hierarchy of reconfigurable interconnects that allow the blocks to be 'wired together'" to perform a required function.

While an ASIC is likely to be faster, its functionality is fixed, whereas an FPGA can be reprogrammed if desired. That could be to fix bugs, add new functions or reimplement existing functions in a more efficient way.

FPGAs can provide higher performance per watt than CPUs or GPUs for certain functions.

So Microsoft uses FPGAs to implement SDN (software defined networking) for Azure.

According to Microsoft Azure CTO Mark Russinovich, processing 40Gbps of traffic in and out of a server would occupy eight full Xeon cores, which are then no longer available to run customers' virtual machines, as well as introducing latency into the network flows.

Microsoft Azure CTO Mark Russinovich

"Microsoft had already started to develop some expertise in FPGAs and when we looked at it we thought maybe these could solve the problem," he said.

ASICs would give the highest performance, lowest cost and lowest power consumption. "But the challenge we had when we considered ASICs is our software defined network continues to evolve and get new features all the time. If we locked ourselves into an ASIC that takes 18 months to two years to come out with, by then our software defined networking requirements would have changed and we would be stuck with something that was two years old and we would have to try to use that one for at least two years to make our investment worthwhile."

FPGAs, on the other hand, can be reprogrammed as required. "About once a month we update our stacks on these FPGAs across the fleet to fix bugs, improve performance and add new features.," he said.

Using FPGAs can reduce latency by an order of magnitude, for example from 500 microseconds to 50 microseconds or lower, he said, "with zero usage of the [CPU] cores.

This technology is able to support 50 and 100Gbps networking, Russinovich suggested.

"This approach lets us adapt more easily, basically futureproofing us, while driving world-class performance," he told iTWire.

FPGAs are also being put to work in Azure's AI services.

Training a deep learning model is very computationally intensive, and so GPUs are almost universally used for that task.

But when it comes to inferencing (eg, getting a trained model to identify the object in a photograph, the Mandarin equivalent of an English sentence, or the sentiment expressed by a piece of text), it can be hard to keep a GPU busy.

Interactive situations call for instant responses. "You don't want to sit there and wait for a batch of those queries to build up before you submit it to the hardware", but that's what is needed to make the most use of a GPU, he said.

"You just want to send in the queries and get them right back. GPUs are just not able to do realtime AI today. But we realised again with our investments in FPGAs we could put in deep learning models and get realtime [responses] back. That's a project called Brainwave."

Russinovich showed a demo that delivered inferencing queries to a GPU in progressively smaller batches, and as the batch size dropped, so did the number of operations performed per second, as a growing proportion of the GPU's resources sat idle.

But with an FPGA-based inferencing implementation, the number of operations remained steady at around 40 teraflops whether queries were delivered in batches of 256 or individually.

"We're pushing the envelope for machine learning infrastructure in Azure," he said, not just by offering the latest and greatest Nvidia GPUs but also by investing in alternative technologies such as FPGAs.

The Brainwave API would soon be available to Azure customers, Russinovich said.


You cannot afford to miss this Dell Webinar.

With Windows 7 support ending 14th January 2020, its time to start looking at your options.

This can have significant impacts on your organisation but also presents organisations with an opportunity to fundamentally rethink the way users work.

The Details

When: Thursday, September 26, 2019
Presenter: Dell Technologies
Location: Your Computer


QLD, VIC, NSW, ACT & TAS: 11:00 am
SA, NT: 10:30 am
WA: 9:00 am NZ: 1:00 pm

Register and find out all the details you need to know below.



iTWire can help you promote your company, services, and products.


Advertise on the iTWire News Site / Website

Advertise in the iTWire UPDATE / Newsletter

Promote your message via iTWire Sponsored Content/News

Guest Opinion for Home Page exposure

Contact Andrew on 0412 390 000 or email [email protected]


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments