Real-time 3D Object Reconstruction

In the last few years Virtual and Augmented Reality technologies have exited the field of pure research and started entering the world of business. Plenty of entrepreneur saw the commercial potentiality of these innovative technologies and have started investing heavily to get ahead of the game and secure a place in this upcoming market.
This shift has greatly increased the resources invested in VR and have brought to a substantial improvement and spread of the technology. More recently then the matter of creating content has become a prominent issue. Multiple companies have started to create completely new environments as well as mapping real-world objects/buildings to 3D models.
Their efforts have brought lots of data to the community, from design objects to famous buildings, many items are available on online stores. This is great for technical application, but indeed limits the amount of personalization that a user can have. Let us consider a user who is interacting with a group of friends in a virtual world. He can show them the route he has taken visiting Manhattan in great details, but he cannot show them the LEGO model that his son has just made in his living room.

That is, until Seene. Seene has just launched a smartphone app that is able to create a 3d model of an object almost in real-time using just the smartphone camera. This means that a user can simply rotate around an object with his smartphone camera to obtain a high-resolution model that can be shared online almost instantaneously.
Seene has achieved this remarkable result by implementing a super-efficient pipeline that fits the data from just one scan to a canonical mesh. The pipeline not only is very fast, but does not require any specialized hardware, making it perfect for smartphones and tablets.

The potentialities of this tool are infinite. Users could personalize their Virtual spaces with object from their very own home.  A game developer could insert some real-world replica of peculiar object in his application almost effortlessly.
But is in the collaboration context that this application really makes the difference. Users could share any objects and create realistic avatars of themselves without previous work or planning. This in turn can make virtual experiences in public places, such as a museum, more realistic and consequently more immersive.

Seene’s application also works on larger objects. For instance, the app is able to create a model of a room just by scanning the wall from the center of it. This capability can, for instance, help a vendor presenting a house to clients who cannot come and visit it, as well as allowing them to personalize it with their own virtual furniture or visualizing any renovation that they may plan to do.

Seene then seems to have revolutionized the 3D scanning market. However, its models, although quite accurate, cannot compete with models obtained by high-resolution laser scanners. For technical application, where dimensions must be accurate and details make the difference, the standard methods are still the only way to go.

Having said this, being able to quickly obtain a 3D model of an object still is a game changer. It represents one of the most important steps to make Virtual and Augmented Reality used every day and we can expect to hear more about Seene as more and more VR and AR applications reach the smartphone.

Nov. 17th, 2017



This article was inspired by the research of Bhargav Arisetty which can be found here.

Note: This article was written as an homework for course CS 491 – Virtual and Augmented Reality, held by professor Johnson at UIC.