XR City Builder Prototype
Client
XR City Builder Prototype
Year
2022
My Scope of Work
UX Design
Project
XR | Tech
The AI-Powered Mixed Reality City Builder project explores how to Collaborate with AI in a Mixed Reality Environment. With a mission to involve every citizen in the city planning process, this project proves just how powerful AI, mixed reality (MR), and co-creation can be in transforming the urban landscape.
My Role
As part of an innovative exploration, Skellefteå Kommun partnered with Science city to research a new way of involving the population in the creation of the upcoming transport centre. Creative Crowd, Substorm and North Kingdom joined the project to propose an AI assisted tool where the user can shape their own version of the building. As UX lead, I was responsible for defining the user journey from the moment a user puts on the headset through the user creation process. I defined the relationship between gestures - building with hand and eye tracking in a MR headset; and voice command, how to utilize the avatar and test the new usage of speech to visual AI, train the model using ChatGPT to have the correct prompt to improvise dialogue with a user during the experience.
Approach
One of the key features of the MR experience is the AI-powered 3D avatar, which acts as an intelligent interface by connecting to chatGPT. This revolutionary AI-assistant plays a crucial role in guiding users through the experience, helping them interact with the digital environment, and effortlessly create digital content for their ideal city block. During the project, users have the opportunity to don a VARJO headset, which places them in an immersive XR environment. The state-of-the-art leap motion SDK technology is employed to detect user gestures, allowing them to physically shape the structure of the transport center while the AI avatar delivers support and insightful suggestions.
Challenges & Solutions
Features
