Wednesday, 28 March 2018 10:18

Nvidia promotes hyperscale inferencing


GPU-accelerated deep learning is increasingly a part of applications running in hyperscale data centres. Nvidia has announced a raft of product improvements and industry collaborations addressing this market.

TensorRT 4 accelerates deep learning inferencing. The new version is said to be up to 190 times faster than conventional CPUs for common applications including computer vision, translation, speech recognition and recommendation engines.

In addition, Nvidia and Google have jointly integrated TensorRT and TensorFlow 1.7, delivering up to eight times greater inference throughput compared with conventional GPU execution.

Tthis [integration] dramatically improves inferencing performance," said vice-president and general manager of accelerated computing Ian Buck.

SAP reports its deep learning-based recommendation application runs 45 times faster on TensorRT with Tesla V100 GPUs than it does on conventional CPUs.

The Kaldi speech framework has been optimised for Nvidia GPUs, allowing faster and more useful virtual assistants, at lower cost to datacentre operators.

GPU acceleration for Kubernetes has been announced by Nvidia in the form of contributions to the open-source project.

Mathworks has announced TensorRT integration with Matlab, giving the ability to generate inference engines from Matlab to Jetson, Nvidia Drive and Tesla.

Nvidia has worked with Amazon, Facebook and Microsoft to ensure software developed using ONNX frameworks including Caffe 2, Chainer and Pytorch can be deployed on Nvidia deep learning platforms.

Nvidia GPU Cloud provides a registry of pre-built containers for running various pieces of GPU-accelerated software in the cloud. "You log in, you download, you run," said founder and chief executive Jensen Huang. Thirty containers are currently available, all certified to run on AWS, Google Cloud, AliCloud and Oracle Cloud, as well as on DGX systems. Azure is still being qualified.

"This is the only architecture that is 'all cloud,'" he said during his keynote address at the GPU Technology Conference.

Huang went on to say that the availability of Kubernetes on Nvidia GPUs "is going to bring joy." The ability to take massive workloads and orchestrate them across hyperscale data centre resources means "life is complete," he joked.

Nvidia demonstrated an image recognition system processing 4.3 images per second on an Intel Skylake-based server. On a single Volta GPU, that rocketed up to 874 images per second. Moving the same job to eight Kubernetes containers, each with a V100 GPU gave around 6900 images per second. The demonstration then failed-over half of those to AWS containers, and not only was the speed maintained, in increased slightly to some 7100 frames per second.

Disclosure: The writer attended Nvidia's GPU Technology Conference as a guest of the company.


26-27 February 2020 | Hilton Brisbane

Connecting the region’s leading data analytics professionals to drive and inspire your future strategy

Leading the data analytics division has never been easy, but now the challenge is on to remain ahead of the competition and reap the massive rewards as a strategic executive.

Do you want to leverage data governance as an enabler?Are you working at driving AI/ML implementation?

Want to stay abreast of data privacy and AI ethics requirements? Are you working hard to push predictive analytics to the limits?

With so much to keep on top of in such a rapidly changing technology space, collaboration is key to success. You don't need to struggle alone, network and share your struggles as well as your tips for success at CDAO Brisbane.

Discover how your peers have tackled the very same issues you face daily. Network with over 140 of your peers and hear from the leading professionals in your industry. Leverage this community of data and analytics enthusiasts to advance your strategy to the next level.

Download the Agenda to find out more


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments