With the new Project Starline prototype, Google may have figured out how to revolutionize video calling through a clever combination of AI and 3D modeling. We were able to try and it’s amazing.
The man in front of me hands me an apple. She seems to be within reach, about 30 centimeters away. I reach out my hand… but I can’t grab the fruit. I know perfectly well why and yet my brain has trouble understanding everything. I’m facing Google’s latest Project Starline prototype and it is simply insane.
On the sidelines of Google I / O 2023, the American giant has indeed unveiled a new version of Project Starline. A still-experimental technology that promises to revolutionize our video calls.
A most surprising video call
Invited to test this experience, I find myself in a small room in one of the many Google buildings scattered around Mountain View. I notice a chair, a table and, above all, a kind of big 65 inch tv (and a very large sound bar) in front of which I am asked to take my place.
The Google spokesperson closes the door behind me, settles in another room further with the same device. He launches a video call with me through interposed screens. Nothing extraordinary… a priori.
My interlocutor, Jason Lawren, co-founder of the project, really seems to be sitting on the other side of the table. Something deep inside me can’t process the fact that Jason isn’t in the same room as me.
I lean to the left, to the right, and I distinguish perfectly the sides of his face, as I could do if he were really a meter away from me. He hands me the famous apple and I have the feeling that if he threw it, it would hit me in the face. He offers me a checkclosed fist against closed fist, and here I am almost surprised not to touch anything.
“A Magical Window”
The 3D effect is stunning, no more no less. Especially since the edges of the television seem to have disappeared and it is the various cameras that frame it that allow me to guess its contours. Immersion is very good, and it takes a few artifacts in the image — Jason’s silhouette ripples at times — to remind me that I’m only seeing a 3D projection of him.
Alas, taking photos and video recordings are prohibited. A person at Google unofficially explains to us that our cameras and smartphones cannot correctly transcribe the 3D experience of Project Starline. So you have to be content with the official visuals and take my word for it.
YouTube linkSubscribe to Frandroid
A question necessarily arises: how does Project Starline work? Google describes a technology that behaves like a magic window “. You might think that magic is a simple hologram. However, what we are really dealing with is artificial intelligence, photorealistic 3D modeling and ultra-efficient real-time processing.
The objective: to improve the quality of exchanges by making them more natural even when two people are in two different cities.
The first version of Project Starline actually dates from 2021. But at the time, the system required much larger hardware with the same size as a “dining booth“. Mean by that the boxes ofdinnersAmerican-style with two large benches and a table.
The brand new prototype that I was able to try is much more compact and promises to be much easier to install in a meeting room for example, or even at home in the more distant future.
A natural conversation
The cameras around the screen capture my movements and the direction of my gaze. An artificial intelligence concocted by Google retrieves this data in real time. She can thus reconstitute a photorealistic 3D image of me to show it to Jason while offering me a perception of my interlocutor faithful to that which I would have had if he were indeed physically in front of me. All depth effects are perfectly transcribed.
That’s what upsets me the most. Technically, I didn’t see a real video of Jason. I saw a live 3D reconstruction. In addition, this notion of real time is essential. Immersion would be totally broken if there was any latency.
Google therefore prides itself on succeeding in compressing the data and interpreting it in the blink of an eye so that the discussion remains very natural. The artifacts mentioned earlier in this article are undoubtedly due to small jolts in the process.
An experimental product
Finally, note that when I leaned too far to the side, a notification on the TV asks me to refocus.
No price or release date has been announced yet. At present, Project Starline is still a slightly crazy experimental product that aims to make a big splash in the years to come. In the meantime, I still haven’t managed to catch the apple.
NB. Our journalist Omar participates in the Google I/O in Mountain View as part of a press trip organized by Google.
The Watt Else newsletter is THE unmissable Numerama event dedicated to the mobility of the future. Register here!