At the Augmented World Expo on Tuesday, Snap unveiled an early version of its real-time on-device image diffusion model that can create vivid AR experiences. The company also unveiled generative AI tools for AR developers.
Bobby Murphy, Snap co-founder and CTO, explained on stage that the model is small enough to run on a smartphone and fast enough to re-render frames in real time, guided by a text prompt.
Murphy said that while the emergence of generative AI image diffusion models is exciting, these models would need to be significantly faster to be effective for augmented reality, which is why his teams have been working on accelerating machine learning models.
Snapchat users will see the first Lenses using this generative model in the coming months, and Snap plans to make it available to developers by the end of the year.
“This and future generative ML models built in real time on devices point to an exciting new direction for augmented reality and give us room to rethink how we render and create AR experiences altogether,” said Murphy.
Murphy also announced that Lens Studio 5.0 is launching today for developers, giving them access to new generative AI tools that will enable them to create AR effects much faster than currently possible, saving weeks and even months.
AR developers can create selfie lenses by generating highly realistic ML facial effects. In addition, they can generate custom stylization effects that apply a realistic transformation to the user’s face, body, and environment in real-time. Developers can also generate a 3D asset in minutes and integrate it into their lenses.
Additionally, using the company’s Face Mesh technology, AR developers can create characters like aliens or wizards with a text or image prompt. They can also create face masks, textures and materials in minutes.
The latest version of Lens Studio also includes an AI assistant that can answer potential questions from AR creators.
