Challenge
Design and develop a Virtual Reality experience embodying user into the vision of a synesthete and their perception of Synesthesia. Wassily Kandinsky was chosen as a reference, since his research findings and documentation on synesthesia propelled the VR experience design decisions. Research of different syntesthetes and their perception of colour to sound. The project needed clear guidelines of shapes, colours, and forms to be used in the experience. Additionally, deep research on sound libraries was required to bridge each colour & shape to sounds and tones of various instruments. For example: yellow colours would playback a range of trumpet notes. Communicating the feeling of Synesthesia to the user inside Virtual Reality as a visual feedback was crucial to ensure the user comprehends seeing and hearing colours.
Deliverables
Guideline on colour to sound relationship created, closely followed by the team while developing the experience. Wassiliy Kandinsky was chosen as a role model of a Synesthete. His research and books on Synesthesia tremendously helped the team in identifying colour palette of the experience as well as the visual style of the virtual world. Color filters were introduced as a visual playback on successful identification of shape and colour. The user would gaze at an object to see a virtual coloured overlay on their eyes approving the seen object.
Process
Researching and gathering requirements to create a sense of synesthesia inside Virtual Reality. Design and prototyping of User Interactions, visual feedback, and virtual environment to embody the user into the mind of Wassily Kandinsky. In this project, I lead a team of developers, 3D designers, and sound engineers to establish guidelines and rules to applying sound to colours connection.
Gallery of Prototypes
Prototyping Interactions
User's interaction inside Virtual Reality was a significant key to a successful embodiment into a synesthete. The team experimented hard in search for an ideal human to computer input in VR such that the user discovers the ability to see like a synesthete.
Experience Walkthrough
Video showcasing the project in production. In this video, real life user interactions and emotions from the experience are captured. Some of the users were later interviewed on their understanding of synesthesia and feelings they had while being a synesthete for couple of minutes in Virtual Reality.