Microsoft is betting that MR is the productivity tool the world needs. MR is a clever mix of holographic images overlayed on the real world. After an amazing, hands-on demonstration, I can see what it means – let games have the VR world, but almost all else is more suited to MR.
Microsoft HoloLens is the first fully untethered, holographic computer running Windows 10. It is completely self-contained, with no wires, phones, or connection needed to a PC.
It was the first time I have used HoloLens for an extended period. In a blog-like manner let me tell you about this incredible experience.
First, let’s eliminate any preconceptions you may have – it is not VR where the entire view is created and rendered on a computer and viewed on a wireless or wired, fully enclosed, cloistered, headset through fisheye lenses from a small screen (usually either a 5+” smartphone or LCD/OLED screen).
You don the HoloLens like a baseball cap. It is a fully self-contained Windows 10 device, where you can still see the world around you, but with the addition of a “white dot” in your field of view that is, in reality, a mouse cursor, showing where you are looking.
The headband weighs 579g, but it does not feel heavy or uncomfortable. Adjust the headband for a firm fit, more to ensure it does not move when you move around. You can even wear glasses. I am long sighted and need reading glasses but I did not need them here as images were crisp and clear.
It has two transparent, polycarbonate lenses (sitting inside the outer visor) that provide amazing, full colour, HD holographic images that have “real” texture and opaqueness – in other words, solid images as if they are there. No, they won't fool you as “real” just yet but when it comes in 4K it might. I was impressed!
I was instructed to look at a table in the corner of the room – and to be sure a Space Shuttle model and a Pokémon style yellow bird appeared. I walked up to them, put my hands through them, and the colours were solid – I could not see my hands beneath. With a single finger gesture (equivalent to a mouse click) the bird sang and danced.
Then there was a demonstration of the solar system with the planets revolving around the sun – it filled the 46m2 room (about 6 x 7m). I could walk around the “solid” image, under it, over it, through it and select — grab — planets, drill down and more. Pretty cool.
Later I saw a demonstration of a real world app by Saab Australia (former car maker and now a defence/security contractor) that set up a war “theatre” on an island replete with full, topographic detail, and all sorts of military intelligence. I saw how to use MR to strategise and rescue hostages. But more impressive was that the two developers were “barely out of nappies” (no offence meant) and took to this medium like ducks to water.
I spent over half an hour immersed in MR and never once felt vertigo or discomfort that VR can cause. I was “safe” in the room with others, and we could discuss, and collaborate normally, each seeing what the other was.
Let’s get technical
The headband contains the HoloLens computer – it is more powerful than an average laptop and has a CPU (central processing unit), GPU (graphics processing unit) and the secret sauce – an HPU (holographic processing unit).
It has an array of sensors to capture where you are looking, movement, gestures, spatial stereo speakers (the red things either side give a precise locational sound without drowning out the real world), battery (about two to three hours use and micro-USB rechargeable), and it communicates wirelessly with a Windows 10 device – a Surface Pro or a desktop is fine.
- 2 x HD, 16:9 lenses capable of 2.5k light points per radiant for realistic, solid holographs;
- 12MP HD video camera to record mixed reality as well;
- 4 environment understanding camera (think Kinect);
- 4 microphones;
- 2 x stereo spatial speakers;
- Human understanding – gaze tracking, gesture input, voice commands;
- Wi-Fi AC; and
- 2GB RAM for CPU, plus 1GB for the HPU and 64GB storage.
And it costs US$3000 for a development edition and US$5000 for a commercial edition. Details here.
What some developers are doing
Case Western Reserve University: “This device allows us to engage students in unprecedented ways,” said Mark Griswold, faculty director for Case Western Reserve’s Interactive Commons and leader of the university’s work with Microsoft HoloLens. “The HoloLens allows students to explore the world in completely new ways. The mixed-reality view means students and faculty can interact with one another and the holographic information the entire time, preserving the critical human connection that is such an essential part of learning. We have just begun to explore the potential of HoloLens, and I am already confident that in time the device will be as common in student backpacks as laptops and smartphones.”
Lowe’s Home Improvement: “At Lowe’s we’ve endeavoured to build a mixed-reality solution to help change the future of our retail experience. When we saw HoloLens, we knew we had found a solution that would allow us to create a collaborative process, enabling customers to make decisions about their home improvement projects quickly and confidently,” said Kyle Nel, executive director of Lowe’s Innovation Labs. “We’re investing the most valuable thing we have: our time and our focus. Microsoft’s deep history of delivering meaningful solutions with other companies assures us that our partnership will help provide a differentiating customer experience.”
thyssenkrupp: “With elevators transporting over one billion people each day, we have a critical role to play in keeping cities moving. At thyssenkrupp, we are focused on leading the much-needed transformation of the global elevator industry to dramatically increase the efficiency and availability of elevators, and HoloLens is a key element in helping us achieve these goals,” said Andreas Schierenbeck, chief executive of thyssenkrupp Elevator. “The application of HoloLens in our operations can reduce service intervention times by up to four times, and such a feat was only made possible through our strong collaboration with Microsoft.”
Airbus: “Airbus believes in mixed-reality technologies, which are already deployed within our products and processes. Microsoft HoloLens is a promising platform, bringing mobility and new ways to consume and link users with digital information. Following our co-development with Microsoft teams, the device is concretely being tested and challenged in various environments across the company to understand if the technology will meet our business expectations. It is a very exciting phase, and we are looking forward to sharing concrete results based on our investigation.”
Audi: “Audi is invested in leading the future of automotive design through the use of cutting-edge technologies. Technology like Microsoft HoloLens could open up new opportunities for our services in many ways-from engineering reviews and collaboration to after sales scenarios and new ways of customer experiences - there are many use cases to be realised," said Jan Pflüger, Co-ordination Augmented- & Virtual Reality at Audi IT. “We innovate to improve service quality, cut time and costs required for maintenance, as well as combining it with a new way of customer communication, a mixed reality solution like HoloLens seems very promising in achieving these goals. We see an exciting future in this technology and look forward to expanding its use at Audi.”
VR is great for games and things requiring a completely immersive experience and MR is for all else. Besides at the introductory price of $5K it is not for the consumer – not just yet anyway.
But I can see advances in this product bringing it to the home in the not too distant future. For example HR resolution and real colours makes it possible to “HoloLens” a 100“ TV set replete with over the air or video content – that could spell the end of that device as we know it. Similarly, computer monitors could be completely replaced by HoloLens.
But its uses in architecture, design, engineering, medicine, military, and commerce are where it will first take hold. Imagine walking into your old kitchen and building/decorating/colour coordinating your dream kitchen from where you stand.
Today marks the commercial availability of HoloLens and like NASA’s Neil Armstrong once said, “One small step for man, one giant leap for mankind.” This is epoch making.