Big Data Market Segment LS
Big Data Market Segment RS
Tuesday, 25 July 2017 00:21

HoloLens 2.0: custom AI coprocessor for deep neural networks coming


When Microsoft releases its second Holographic Processing Unit or HPU, which presumably will arrive in HoloLens 2.0, it will have a custom silicon AI coprocessor for superior mixed reality.

These days, our actual reality is no longer enough. We’ve been going virtual for decades, and now augmented reality has come to life, so much so that we now talk about mixed realities.

Technological evolution is a reality of life, and so a sneak peek into the future of the HPU or holographic processing unit used in Microsoft’s HoloLens wireless headset has been granted to us all by Marc Pollefeys, Microsoft’s director of Science for HoloLens, in a new blog post entitled “Second version of HoloLens HPU will incorporate AI coprocessor for implementing DNNs.”

A DNN is a “deep neutral network”, which can be plural as in DNNs, with Pollefeys noting that it isn’t “an exaggeration to say that deep learning has taken the world of computer vision, and many other recognition tasks, by storm,” with “many of the most difficult recognition problems have seen gains over the past few years that are astonishing".

Pollefeys notes that while we’ve “seen large improvements in the accuracy of recognition as a result of Deep Neural Networks, deep learning approaches have two well-known challenges: they require large amounts of labelled data for training, and they require a type of compute that is not amenable to current general purpose processor/memory architectures".

He notes that “some companies have responded with architectures designed to address the particular type of massively parallel compute required for DNNs, including our own use of FPGAs, for example, but to date these approaches have primarily enhanced existing cloud computing fabrics".

Pollefeys then states that he works “on HoloLens, and in HoloLens, we’re in the business of making untethered mixed reality devices. We put the battery on your head, in addition to the compute, the sensors, and the display. Any compute we want to run locally for low-latency, which you need for things like hand-tracking, has to run off the same battery that powers everything else. So what do you do?”

The answer is that “You create custom silicon to do it,” with this video then displayed.


Pollefeys goes on to give additional background on the HPU, with all the tech and sensors that go in to making the HPU, which inside HoloLens makes it “the world’s first — and still only — fully self-contained holographic computer".

We then get some extra detail on how Microsoft’s Harry Shum, the executive vice-president of the company’s “Artificial Intelligence and Research Group, announced in a keynote speech at CVPR 2017, that the second version of the HPU, currently under development, will incorporate an AI coprocessor to natively and flexibly implement DNNs".

On display was an “an early spin of the second version of the HPU running live code implementing hand segmentation”, with a range of layer types that are fully programmable by Microsoft’s HoloLens creators.

Pollefeys concludes by stating that “The AI coprocessor is designed to work in the next version of HoloLens, running continuously, off the HoloLens battery.

“This is just one example of the new capabilities we are developing for HoloLens, and is the kind of thing you can do when you have the willingness and capacity to invest for the long term, as Microsoft has done throughout its history.

“And this is the kind of thinking you need if you’re going to develop mixed reality devices that are themselves intelligent. Mixed reality and artificial intelligence represent the future of computing, and we’re excited to be advancing this frontier.”

So, although it is obviously no secret that a successor version is under development, we’ve been given a sneak peek into the advancements a second-gem HPU will deliver – and it is no doubt keeping Apple, Google, Facebook and Oculus, HTC, Samsung, Magic Leap and everyone else on their toes with developments of their own yet to come!


You cannot afford to miss this Dell Webinar.

With Windows 7 support ending 14th January 2020, its time to start looking at your options.

This can have significant impacts on your organisation but also presents organisations with an opportunity to fundamentally rethink the way users work.

The Details

When: Thursday, September 26, 2019
Presenter: Dell Technologies
Location: Your Computer


QLD, VIC, NSW, ACT & TAS: 11:00 am
SA, NT: 10:30 am
WA: 9:00 am NZ: 1:00 pm

Register and find out all the details you need to know below.



iTWire can help you promote your company, services, and products.


Advertise on the iTWire News Site / Website

Advertise in the iTWire UPDATE / Newsletter

Promote your message via iTWire Sponsored Content/News

Guest Opinion for Home Page exposure

Contact Andrew on 0412 390 000 or email [email protected]


Alex Zaharov-Reutt

One of Australia’s best-known technology journalists and consumer tech experts, Alex has appeared in his capacity as technology expert on all of Australia’s free-to-air and pay TV networks on all the major news and current affairs programs, on commercial and public radio, and technology, lifestyle and reality TV shows. Visit Alex at Twitter here.



Recent Comments