Our inside look at Microsoft’s HoloLens leaves us wanting more

Microsoft’s HoloLens, a headset that floats 3D images in front of your eyes and overlays them on top of real-world environments, was the show-stealer of the company’s Build developer conference this week. That’s despite Microsoft having shared virtually zero new information since the January unveiling regarding how HoloLens works and when it may arrive fully formed.
Yet after experiencing it for the second time in a more realized prototype Thursday, I can say with confidence that the HoloLens is no gimmick. I would buy one now if I could, and if there were any Windows apps that ran on it. Though its release date is a mystery and the apps nonexistent (for now), the HoloLens has the potential to be the first truly successful so-called augmented-reality device — a distinction that cannot be understated.
Microsoft is making a gamble that the future of computers will be devices that mix natural ways of interacting with the world — voice, gestures and sight — into tools for creating, consuming and communicating digitally. It’s even tailored its upcoming Windows 10 operating system to be the software that powers everything and anything with or without a screen. Unlike devices such as the Oculus Rift virtual-reality headset, which immerses us in 360-degree worlds, the HoloLens wants to blend virtual spaces with real ones — hence, augmenting them.
To make sure we understood the many ways in which the device could become a new computing platform, Microsoft sat dozens of us down in a large hotel room with desktops running Windows 10. The mission: Help us, step-by-step, develop a rudimentary holographic app using software from game toolkit maker Unity.
Back in January, Microsoft had us wearing crude prototypes that resembled optometrist gear, with wires running out and a heavy pack of components worn around the neck. This go-round, Microsoft had hundreds of fully enclosed, finished prototypes that look just like the version it had on display in its Wednesday keynote presentation.
As was the case in January, the company would not allow photography or video of any portion of the demo.
To demonstrate how developers will create apps for the Windows Holographic platform, Microsoft took us through a series of educational demos by loading code directly onto the HoloLens for each one of the device’s key features.
Gaze Control
The mosbt noticeable differentiator with the HoloLens right now is that your field of view is limited, resembling a rectangle the size of a sheet of printer paper held a half a foot in front of your face. That means that none of the 3D objects are picked up in your peripheral vision and they tend to clip off at the edges as you walk around.
That became clear during the first few portions of the demo, when we organized a static platform with floating origami spheres in Unity with a handful of premade assets. Without the HoloLens being able to detect where we were looking, the platform was only viewable in one distinct location and only when we looked at it just right once we put the headset on.
Because the HoloLens can understand the environment — it has motion cameras and sensors built in — the platform didn’t follow our line of sight, but rather stayed fixed in one location.
After activating the so-called gaze control, however, we were able to direct our line of sight, represented by a red circle, at each distinct object in the scene. (Microsoft had prewritten every coding script in the demo, so this meant we checked a box in Unity to turn it on). Gaze control is one of the five pillars of the HoloLens because it most closely resembles a computer’s mouse cursor.
Gesture Control
The next most important feature in Microsoft’s toolkit is gesture control, which most closely resembles a mouse click.
When turning on gesture control and adding physics to the spheres in Unity (more checked boxes), we were able to use a simple tapping motion with our pointer finger and thumb to make the spheres fall and land on the surface of the platform. Once they rolled off, however, they were gone forever — until you reset the program that is.
Don’t expect any Minority Report-style full-hand motions any time soon.
Air Tap, as the gesture is called, is the HoloLens’ only physical interaction tool as of now. Don’t expect any Minority Report-style full-hand motions any time soon. From my experience , it’s difficult enough to make sure the HoloLens recognizes even a gesture this simple, and Microsoft won’t say yet whether we’ll see anything as complex as pinch-to-zoom with the HoloLens.
Voice Control
The HoloLens is an entire computer enclosed within a single headset, meaning it has a battery, a CPU, a graphics processor and what Microsoft calls a holographic processing unit, or HPU. The device also has speakers for putting out sound directly next to your ear and microphones for picking up your voice.
When we enabled voice control, we could talk to the HoloLens through spoken commands as one would speak to Apple’s Siri or Microsoft’s Cortana software assistants. “Drop the sphere” made the sphere fall, while “reset” let us try again. It’s worth noting how bizarre it was to watch an entire room full of people wearing holographic headsets, mumbling phrases to themselves and furiously tapping the air.
Spatial sound
Spatial sound is one of the more interesting HoloLens features because it re-creates the sensation of sound coming from a distinct location, which is instrumental in making virtual objects and sensations feel more real.
Small red speakers resting near the wearer’s ears let the HoloLens pump out spatial sound, which is audio designed to replicate how we hear sound traveling from its source.Nate Ralph/CNET
Once we activated spatial sound, a serene tune filled the room in a subtle fashion, so discreetly in fact that I told the Microsoft employee walking me through the demo that it wasn’t working at first. The truth was that it was quietly playing as if it were being generated across the room, where the floating platform had been placed.
As I moved closer, the audio became louder. When I forced the origami spheres to drop, they made gentle crumpling sounds as if paper were hitting a real physical surface.
Spatial mapping
Microsoft saved the best for last: spatial mapping.
Because the HoloLens is packed to the brim with sensors and cameras that help it see and understand the environment, we were able to turn everything around us within a certain focal distance into a wire-frame object, letting us see the world as the HoloLens perceives it.
The HoloLens is an entire computer enclosed within a headset, meaning it has a battery, a CPU, GPU and what Microsoft calls a holographic processing unit, or HPU.Nate Ralph/CNET
From there, we could place our platform anywhere we liked, so long as the HoloLens could map the object and create a concrete wire frame for it. To finish off the demo, we added one final asset to the platform that, once touched by one of the falling spheres, would blow a hole in space. Inside was a cartoonish cavern filled with clouds and flying dinosaurs, like a scene straight out of a Disney film.
Once the 90 minutes were up, we plugged back in the micro-USB cables connecting the HoloLens units with the Windows PCs and were escorted out, having become momentary holographic developers, in Microsoft parlance.
As we left behind Microsoft’s most top-secret gadget, walking past the signs barring the use of cameras and smartphones, the takeaway was simple: I want one.
Source: Cnet