Skating with Quest Pro – Meta Presence Hack '23

Skating with Quest Pro on – Meta Presence Hack '23

Overview

Role:
Developer

Year:
2023

Objective:
The primary objective of the Skating with Quest Pro on project was to create an innovative mixed reality (MR) game using Meta’s Presence platform during the Meta Presence Platform Hackathon '23. The game aimed to teach skateboarding while promoting environmental awareness by integrating real-world interactions with virtual elements, utilizing the advanced capabilities of the Meta Quest Pro headset.

Technologies Used:

• Meta Quest Pro

• Meta Presence Platform

• Facial tracking technology

• Text-to-speech

Collaborators:
Ash Maddi, Charles “Al” Baker, Elena Piech, Paige Dansinger

The Challenge

Developing an MR game that allowed users to skateboard while safely interacting with the game presented unique challenges. User safety was a top priority since wearing a helmet was not compatible with the Quest Pro headset. Another challenge was finding an intuitive way for users to interact with the game while skateboarding, without using traditional controllers.

System Design

Game Concept: Joni Apple Skate

"Joni Apple Skate" is a mixed reality game designed to teach skateboarding and promote green awareness. The player skates in the real world while interacting with the virtual environment through facial gestures. Players collect coins by leaning left or right on their skateboard, and with each coin collected, virtual plants grow in their real environment, merging education with physical activity.

Facial Detection for Interaction

Since traditional controllers couldn't be used safely while skateboarding, we utilized facial detection technology. Players start the game with a wink, and other facial gestures are used for interactions. This feature enabled hands-free gameplay while keeping the game safe and responsive.

Safety and User Experience

Due to safety concerns, wearing a helmet with the Quest Pro was not feasible. We pivoted the project to ensure the gameplay interactions were hands-free and safe. The design prioritized intuitive gestures and minimal physical interference, allowing the user to focus on skateboarding while still engaging with the game.

Environmental Awareness

The game incorporated an environmental theme where users’ actions, such as collecting virtual coins, promoted the growth of virtual plants in the real world. This feature combined education with entertainment, raising awareness about green practices while enhancing the skateboarding experience.

Key Responsibilities

Development

As a developer, my primary responsibility was to integrate the facial tracking technology with the gameplay mechanics, ensuring smooth, intuitive interactions. I focused on leveraging the Quest Pro’s advanced capabilities to create an immersive and seamless MR experience that felt natural and responsive during skateboarding. We also used the controllers as trackers to get the position and the rotation of the physical skateboard.

Project Highlights

Innovative Interaction

The use of facial detection for hands-free interaction was one of the standout features of the game. This allowed users to control the game in a way that was both safe and intuitive while skateboarding, showcasing the Quest Pro’s advanced capabilities.

Educational Gameplay

In addition to providing an engaging skateboarding experience, the game integrated an educational element. By promoting environmental awareness through the virtual growth of plants, players were encouraged to think about sustainable practices while enjoying the game.

User-Centric Design

The focus on user-centric design ensured that the game was easy to interact with and enjoyable. The user feedback we received was positive, with players appreciating the new interaction system and the overall immersive experience.

Results

The Skating with Quest Pro on game successfully demonstrated the potential of mixed reality technology for creating innovative, educational, and safe experiences. The project highlighted the importance of hands-free interactions and how facial detection technology can enhance gameplay in active environments like skateboarding.

Learnings

This project provided valuable insights into using facial tracking technology in MR games and designing intuitive, safe interactions. I learned the importance of balancing immersive gameplay with practical safety considerations, especially in active physical environments.

Future Enhancements

Future improvements could include:

• Expanding the range of facial gestures for more interactive gameplay.

• Adding more advanced environmental features, such as dynamic weather effects or more complex interactions with the virtual plants.

• Further optimizing the game for upcoming MR devices, expanding compatibility beyond the Quest Pro.