Wednesday, 28 March 2018 10:18

Nvidia promotes hyperscale inferencing


GPU-accelerated deep learning is increasingly a part of applications running in hyperscale data centres. Nvidia has announced a raft of product improvements and industry collaborations addressing this market.

TensorRT 4 accelerates deep learning inferencing. The new version is said to be up to 190 times faster than conventional CPUs for common applications including computer vision, translation, speech recognition and recommendation engines.

In addition, Nvidia and Google have jointly integrated TensorRT and TensorFlow 1.7, delivering up to eight times greater inference throughput compared with conventional GPU execution.

Tthis [integration] dramatically improves inferencing performance," said vice-president and general manager of accelerated computing Ian Buck.

SAP reports its deep learning-based recommendation application runs 45 times faster on TensorRT with Tesla V100 GPUs than it does on conventional CPUs.

The Kaldi speech framework has been optimised for Nvidia GPUs, allowing faster and more useful virtual assistants, at lower cost to datacentre operators.

GPU acceleration for Kubernetes has been announced by Nvidia in the form of contributions to the open-source project.

Mathworks has announced TensorRT integration with Matlab, giving the ability to generate inference engines from Matlab to Jetson, Nvidia Drive and Tesla.

Nvidia has worked with Amazon, Facebook and Microsoft to ensure software developed using ONNX frameworks including Caffe 2, Chainer and Pytorch can be deployed on Nvidia deep learning platforms.

Nvidia GPU Cloud provides a registry of pre-built containers for running various pieces of GPU-accelerated software in the cloud. "You log in, you download, you run," said founder and chief executive Jensen Huang. Thirty containers are currently available, all certified to run on AWS, Google Cloud, AliCloud and Oracle Cloud, as well as on DGX systems. Azure is still being qualified.

"This is the only architecture that is 'all cloud,'" he said during his keynote address at the GPU Technology Conference.

Huang went on to say that the availability of Kubernetes on Nvidia GPUs "is going to bring joy." The ability to take massive workloads and orchestrate them across hyperscale data centre resources means "life is complete," he joked.

Nvidia demonstrated an image recognition system processing 4.3 images per second on an Intel Skylake-based server. On a single Volta GPU, that rocketed up to 874 images per second. Moving the same job to eight Kubernetes containers, each with a V100 GPU gave around 6900 images per second. The demonstration then failed-over half of those to AWS containers, and not only was the speed maintained, in increased slightly to some 7100 frames per second.

Disclosure: The writer attended Nvidia's GPU Technology Conference as a guest of the company.


As part of our Lead Machine Methodology we will help you get more leads, more customers and more business. Let us help you develop your digital marketing campaign

Digital Marketing is ideal in these tough times and it can replace face to face marketing with person to person marketing via the phone conference calls and webinars

Significant opportunity pipelines can be developed and continually topped up with the help of Digital Marketing so that deals can be made and deals can be closed

- Newsletter adverts in dynamic GIF slideshow formats

- News site adverts from small to large sizes also as dynamic GIF slideshow formats

- Guest Editorial - get your message out there and put your CEO in the spotlight

- Promotional News and Content - displayed on the homepage and all pages

- Leverage our proven event promotion methodology - The Lead Machine gets you leads

Contact Andrew our digital campaign designer on 0412 390 000 or via email



Security requirements such as confidentiality, integrity and authentication have become mandatory in most industries.

Data encryption methods previously used only by military and intelligence services have become common practice in all data transfer networks across all platforms, in all industries where information is sensitive and vital (financial and government institutions, critical infrastructure, data centres, and service providers).

Get the full details on Layer-1 encryption solutions straight from PacketLight’s optical networks experts.

This white paper titled, “When 1% of the Light Equals 100% of the Information” is a must read for anyone within the fiber optics, cybersecurity or related industry sectors.

To access click Download here.


Stephen Withers

joomla visitors

Stephen Withers is one of Australia¹s most experienced IT journalists, having begun his career in the days of 8-bit 'microcomputers'. He covers the gamut from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences and a PhD in Industrial and Business Studies.



Recent Comments