In 2018, Facebook introduced a trivial, but nice feature: the ability to capture and share 3D photos. In reality, these last ones made it possible above all to add a relief effect to a static shot, by modifying the perspectives of the image when the news feed was unrolled. A fun novelty, but reserved for almost two years for smartphones with two sensors and therefore capable of taking photos in portrait mode. A limit which today tends to fade thanks to the magic of AI.
Cheating with depth data
Indeed, the creation of these 3D photos required the use of two photo modules, in order to take a picture from two different angles to combine them and obtain the desired relief effect. Enough to put the functionality out of reach of smartphones equipped with a single sensor.
To remedy the problem and offer the functionality to more users, Facebook relies on AI to artificially "convert" a 2D photo into a 3D snapshot, even without portrait mode. To achieve this, Engadget explains, the firm uses its neural networks to “estimate the distance of each pixel from the camera”. A plan B which makes it possible to replace the depth data so far provided by a second photo sensor.
Interesting spinoffs in AR and VR?
The work of Facebook is reminiscent of the method used by Google to allow a portrait mode to its Pixel smartphones. Except that Facebook targets photos that have already been taken in 2D. The company explains that its technique can convert any photo into 3D, whether it's a selfie ... or a snapshot taken "decades ago" and stored on your phone.
Facebook's research could also have interesting spinoffs in the areas of augmented reality and virtual reality, while the group also hopes to benefit, in the long term, the videos.
Be careful, however! All smartphones with single sensors are not yet affected by this novelty, which is currently being deployed. While it is indeed possible to view 3D photos on any device, publishing them remains reserved for certain smartphones only. Facebook explains that only iPhone 7 (or newer) are capable of it, just like “mid-range or higher” Android smartphones. The Galaxy Note 8, Note 9 and the entire Pixel lineup are part of the bundle, with the exception of the Pixel 3A.
