Holoportation

AR Communication

Virtual and Augmented reality technologies are in a boom. Plenty of companies are coming up with new products that try to exploit the technologies in different fields.
Along this line, Microsoft recently unveiled a new communication medium based on AR: Holoportation.

Holoportation is a system that, by using multiple cameras, is able to create an extremely accurate model of a scene and stream it in real-time to an augmented reality device (HoloLens) connected on the internet. The key here is the “real-time” bit. There are multiple technologies already available to create high resolution 3D models of one or multiple objects, but all of them take long times. By making it real-time Holoportation is in fact able to create a continuous stream of 3D models. The HoloLens user can then see the original scene evolve through time in his “reality”.

Leaving apart the technical achievement, Holoportation shows its real potentiality when used in conjunction with audio communication. The most basic implementation we can think of is a sort of AR telephone. Imagine calling someone and seeing them just across the room while you talk. It will revolutionize the business.

Holoportation works using multiple 3D cameras that continuously take pictures of the target space, identify the relevant objects, e.g. humans, merge their content together and obtain a high-resolution 3D model in real-time. The model is then compressed and streamed to a HoloLens, where it is displayed in the augmented scene. In theory two cameras are enough to create a 3D model, but realistically at least 4 or 5 cameras are required to obtain an acceptable model. Moreover, each camera requires a high-speed connection to the others and a calibration process is required.

There are then some steps to do before Holoportation becomes the AR Facetime! Realistically, remote conferencing could be one of the primary beneficiary of this innovative technology. Discussing ideas without the limit of a screen, as big as it may be, and visualizing all the people you are interacting with in the same space you are in can make communication more effective and in turn contribute to improved decision making without the need for travelling. At the same time, videoconference rooms are diffused in many offices and an investment in an upgrade for Holoportation could make economic sense.

Steps are on the way to make the technology more portable. Recently Microsoft has managed to port the whole setup on a car, optimizing the compression and managing to stream with just Wi-Fi connectivity, but again it was a prototype setup and Wi-Fi connectivity imposes serious limitations on the movements of the vehicle.

Overall then, the Holoportation is at the first step of a stairs that is bound to revolutionize our communication standards. Although there are issues to address, Microsoft is definitely on the right way and I believe we can expect to see all our long distance friends sitting on our couch in a few years!

Nov. 12th, 2017

Chicago

Note: This article was written as an homework for course CS 491 – Virtual and Augmented Reality, held by professor Johnson at UIC.