What you can see in this video that has been published recently on Twitter it is not someone’s fast camera, nor is it a drone flight recording video with agility. It is a three-dimensional stage, created by a rendering neural network from a few still photos. At the beginning of 2020 we saw something similar, but the improvement compared to then is enormous.
Some photos from there, others from there and we already have the scene set up
ADOP: Approximate Differentiable One-Pixel Point Rendering
abs: https://t.co/npOqsAstAx pic.twitter.com/LE4ZdckQPO
– AK (@ ak92501) October 14, 2021
As you can see in the video, the detail is tremendous (especially in the letters on the ship’s license plate at 1:18). The engine is called ADOP (Approximate Differentiable One-Pixel Point Rendering) and is exposed on the Cornell University ArXiv website, from where they detail that the scenario is generated from arbitrarily taken photographs. An additional engine is responsible for modifying the tonality of each photograph so that the result is uniform.
Of each photograph variables such as white balance or camera exposure are taken into account, and of course the position and direction the target is pointing. ADOP is capable of managing more than 100 million points in real time. It is something similar to what Denis Shiryaev achieved with the frames of the video described as the oldest ever recorded:
Rodolfo Rosini, founder of ConceptionX investor, has spread the video mentioning some possible applications of this technology. With playable scenarios at such high resolutions think about replicate large-scale concerts or events, or create realistic scenarios in video games or movies.
We can also think about the possibilities of these motors using photographs taken with advanced smartphones, with LiDAR sensors. We could be able to generate a navigable three-dimensional scene of what we want based on a few photographs taken around. OR como says Rosini, “to be able to rotate the angle of the photos” that we have in our reel. Maybe Google will be interested in this technology for future versions of Street View?