Friday, 06 April 2018 17:51

FPGAs play an interesting role in Azure


You might be surprised to learn the part FPGAs are playing in Azure's hardware.

Field-programmable gate arrays (FPGAs) can be seen as an intermediate step between conventional CPUs and ASICs (application specific integrated circuits).

To quote Wikipedia, "FPGAs contain an array of programmable logic blocks, and a hierarchy of reconfigurable interconnects that allow the blocks to be 'wired together'" to perform a required function.

While an ASIC is likely to be faster, its functionality is fixed, whereas an FPGA can be reprogrammed if desired. That could be to fix bugs, add new functions or reimplement existing functions in a more efficient way.

FPGAs can provide higher performance per watt than CPUs or GPUs for certain functions.

So Microsoft uses FPGAs to implement SDN (software defined networking) for Azure.

According to Microsoft Azure CTO Mark Russinovich, processing 40Gbps of traffic in and out of a server would occupy eight full Xeon cores, which are then no longer available to run customers' virtual machines, as well as introducing latency into the network flows.

Microsoft Azure CTO Mark Russinovich

"Microsoft had already started to develop some expertise in FPGAs and when we looked at it we thought maybe these could solve the problem," he said.

ASICs would give the highest performance, lowest cost and lowest power consumption. "But the challenge we had when we considered ASICs is our software defined network continues to evolve and get new features all the time. If we locked ourselves into an ASIC that takes 18 months to two years to come out with, by then our software defined networking requirements would have changed and we would be stuck with something that was two years old and we would have to try to use that one for at least two years to make our investment worthwhile."

FPGAs, on the other hand, can be reprogrammed as required. "About once a month we update our stacks on these FPGAs across the fleet to fix bugs, improve performance and add new features.," he said.

Using FPGAs can reduce latency by an order of magnitude, for example from 500 microseconds to 50 microseconds or lower, he said, "with zero usage of the [CPU] cores.

This technology is able to support 50 and 100Gbps networking, Russinovich suggested.

"This approach lets us adapt more easily, basically futureproofing us, while driving world-class performance," he told iTWire.

FPGAs are also being put to work in Azure's AI services.

Training a deep learning model is very computationally intensive, and so GPUs are almost universally used for that task.

But when it comes to inferencing (eg, getting a trained model to identify the object in a photograph, the Mandarin equivalent of an English sentence, or the sentiment expressed by a piece of text), it can be hard to keep a GPU busy.

Interactive situations call for instant responses. "You don't want to sit there and wait for a batch of those queries to build up before you submit it to the hardware", but that's what is needed to make the most use of a GPU, he said.

"You just want to send in the queries and get them right back. GPUs are just not able to do realtime AI today. But we realised again with our investments in FPGAs we could put in deep learning models and get realtime [responses] back. That's a project called Brainwave."

Russinovich showed a demo that delivered inferencing queries to a GPU in progressively smaller batches, and as the batch size dropped, so did the number of operations performed per second, as a growing proportion of the GPU's resources sat idle.

But with an FPGA-based inferencing implementation, the number of operations remained steady at around 40 teraflops whether queries were delivered in batches of 256 or individually.

"We're pushing the envelope for machine learning infrastructure in Azure," he said, not just by offering the latest and greatest Nvidia GPUs but also by investing in alternative technologies such as FPGAs.

The Brainwave API would soon be available to Azure customers, Russinovich said.


26-27 February 2020 | Hilton Brisbane

Connecting the region’s leading data analytics professionals to drive and inspire your future strategy

Leading the data analytics division has never been easy, but now the challenge is on to remain ahead of the competition and reap the massive rewards as a strategic executive.

Do you want to leverage data governance as an enabler?Are you working at driving AI/ML implementation?

Want to stay abreast of data privacy and AI ethics requirements? Are you working hard to push predictive analytics to the limits?

With so much to keep on top of in such a rapidly changing technology space, collaboration is key to success. You don't need to struggle alone, network and share your struggles as well as your tips for success at CDAO Brisbane.

Discover how your peers have tackled the very same issues you face daily. Network with over 140 of your peers and hear from the leading professionals in your industry. Leverage this community of data and analytics enthusiasts to advance your strategy to the next level.

Download the Agenda to find out more


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments