HandSpell
A turn based battle game where players use real time hand gestures to cast spells and fight enemies, serving as a demo of integrating Python MediaPipe with Unity via OSC.
About the Game
HandSpell is a turn based battle game where players use hand gestures to shape magic and fight enemies. Each gesture corresponds to a different spell, turning your own hand into the core input system. You must figure out the correct hand gestures for each spell by interpreting clues presented through a spell book. The game is a demo of real time hand tracking by connecting Python MediaPipe with Unity through OSC.
Design Goals
The core design goal behind HandSpell was to explore how hand tracking could become a primary game mechanic. The idea originated from a collaborative experiment with a friend using MediaPipe, where we wanted to test how hand gestures might translate into player actions inside a game. I was inspired by Mage Arena, where speaking spell names to cast turns the entire lobby into a chaotic yet playful experience.
Because traditional mouse and keyboard controls were not viable while using hands as input, the game naturally evolved into a turn based combat system inspired by Pokémon battles. This structure allows players to focus on forming gestures without thinking about controls. I designed a puzzle like spell book that shows visual clues, encouraging players to figure out the correct hand gesture for each spell rather than being explicitly told.
Challenges
While the current version of the game is intentionally small in scope and somewhat repetitive, it primarily serves as a technical and design demo for hand tracking gameplay. Future additions such as mana management, deeper spell interactions, and multiplayer features would significantly expand its depth and replayability.