Piano Tiles is an interactive installation developed in TouchDesigner, where players trigger piano tones by touching colored tiles. The game is projected onto a wall and offers both visual and musical feedback.
Meet Audaura — the new way of exploring libraries by ear: a wearable sonic interface that guides you to themes, surfaces librarian picks, and lets stories spread socially. Discover, listen, and find books through spatial audio and playful interactions.
Eavesdropper
Sonically shows what other users of the system are listening to. Fosters connection and socialising.
Theme finder
Guides visitors through the library with calm 8D soundscapes, creating a sonic experience while walking through a library.
Book recommendations
Presents librarian favorites. Provides a sonic platform for librarians to give book recommendations to every user of the system. Guides visitors toward new reading adventures.
Product title
Aut aliquam et at quod. Fuga atque qui. Ullam facere quibusdam. Dolorem ut id. Eum ullam laborum et. Aliquam voluptate ut.
Product title
Product information, Lorem Ipsum. Fuga atque qui. Ullam facere quibusdam. Dolorem ut id. Eum ullam laborum et. Aliquam voluptate ut.
Product title
Product information, Lorem Ipsum. Fuga atque qui. Ullam facere quibusdam. Dolorem ut id. Eum ullam laborum et. Aliquam voluptate ut.
Product title
Aut aliquam et at quod. Fuga atque qui. Ullam facere quibusdam. Dolorem ut id. Eum ullam laborum et. Aliquam voluptate ut.
Product title
Product information, Lorem Ipsum. Fuga atque qui. Ullam facere quibusdam. Dolorem ut id. Eum ullam laborum et. Aliquam voluptate ut.
Product title
Product information, Lorem Ipsum. Fuga atque qui. Ullam facere quibusdam. Dolorem ut id. Eum ullam laborum et. Aliquam voluptate ut.
Experience sound from all around you, with each book’s story coming from a different direction. Navigate the library intuitively and discover your next read with perfect sonic guidance.
We created a scaled-down model of the Malmö City Library, 3D printed miniature versions of our five group members, and provided users with headphones. Visitors could “walk” through the miniature library and experience Audaura firsthand, exploring the sonic layers and interactive features of our design in a playful, hands-on way.
Experience Audaura in action through our prototype video. See how users navigate the miniature Malmö City Library, explore thematic soundscapes, and interact with the sonic layers, bringing our design concept to life in a playful and immersive way.
For this project, I was responsible for the technical development and implementation of the system. This included setting up the tracking, audio routing, and real-time processing that made the prototype functional. Below is a brief explanation of how the final system worked in practice, demonstrating how visitors’ movements and interactions influenced the sonic experience
Our 8D audio design was inspired by sound in games, where directional audio has long been used to guide exploration. We adapted this concept to the library, letting visitors navigate and discover books through immersive, spatial soundscapes.
We used Google’s Mediapipe machine learning model to track the miniature figures’ positions and rotations in real time. This allowed the system to accurately calculate each figure’s orientation, feeding precise spatial data into the audio engine to create an immersive and responsive 8D sound experience.
Using a live video stream fed into TouchDesigner, Google’s Mediapipe tracked the positions and rotations of the miniature figures in real time. TouchDesigner then calculated each figure’s relative location to the different sound sources in the miniature library, in combination with the relative rotation and actual rotation. These signals were then send to Ableton.
These tracking signals were received real time in Ableton, where the direction of each audio source was dynamically adjusted using Sennheiser’s Dear Reality software. Afterwards this was send to two headphones, that were used during the exposition. This created an immersive 8D audio experience, allowing multiple players to explore the soundscape simultaneously
Since November 2022, an initiative has aimed to educate Dutch secondary school teachers and leaders about the opportunities and risks of AI, particularly tools like ChatGPT. The workshops highlighted digital literacy as a core skill, exemplified by a presentation to Minister of Education Mariëlle Paul.
Teaching students to use AI prepares them for the future, requiring teacher training programs to strengthen digital skills.Events include a successful two-day AI program for the NUOVO school group, involving 130 students and 20 teachers, offering hands-on experience with tools like ChatGPT, Leonardo, and Adobe Firefly. Ethics and source attribution were emphasized, supporting a future-ready generation.
An AI platform for teachers is also in development to facilitate knowledge sharing and collaboration. Brainstorming sessions continue to keep teachers engaged in sharing practical applications. These efforts raise awareness of AI’s role in education and the importance of digital skills for students and teachers alike.
.jpeg)
I develop and deliver workshops and training sessions for teachers and school leaders to introduce them to AI and its possibilities. These sessions help them integrate AI tools effectively into their teaching and enhance students’ digital literacy.
I conduct workshops at AI-related events, such as the two-day AI program for the NUOVO school group, where both teachers and students gain hands-on experience with AI tools like ChatGPT. These events provide a platform for knowledge sharing and discussions about the opportunities and ethical issues of AI in education.
On March 17, 2023, we were invited by the VVD parliamentary group in The Hague to speak about the importance of AI education in secondary schools. The information evening in the House of Representatives provided a valuable opportunity to highlight the significance of digital literacy and AI awareness in education, and to discuss the future role of technology in schools.
The launch of ChatGPT immediately highlighted how revolutionary AI can be, including for education. From that moment, I realized that schools needed education to teach students and teachers to use this technology responsibly.
The first workshop for teachers became a reality. In this practical session, education professionals gained insight into the opportunities and risks of AI, and how to use AI tools like ChatGPT responsibly in their lessons.
We were invited to speak at an information evening hosted by the VVD in the House of Representatives on AI in education. There, I shared my vision on the necessity of digital literacy as a core skill in education and the importance of AI awareness for a future-proof generation.
The use of cutting-edge materials and manufacturing processes in the design and production of Elements_Efi has resulted in a product that is both durable and sustainable, reducing the environmental impact of its use.
The use of cutting-edge materials and manufacturing processes in the design and production of Elements_Efi has resulted in a product that is both durable and sustainable, reducing the environmental impact of its use.
The use of cutting-edge materials and manufacturing processes in the design and production of Elements_Efi has resulted in a product that is both durable and sustainable, reducing the environmental impact of its use.
The use of cutting-edge materials and manufacturing processes in the design and production of Elements_Efi has resulted in a product that is both durable and sustainable, reducing the environmental impact of its use.
In addition to workshops and policy advocacy, I also share my experiences with AI education through talks and publications. On March 13, 2025, I was a guest on the HKU and AI Podcast by Mikal van Leeuwen, where we discussed the role of AI in creative education and how AI literacy is promoted in secondary schools. Listen to the podcast below.
Listen to Podcast (Dutch)onderwijs%20-%20Episode.jpg)
Piano Tiles is an interactive installation developed in TouchDesigner, where players trigger piano tones by touching colored tiles. The game is projected onto a wall and offers both visual and musical feedback.
Piano Tiles is an interactive installation where movement, sound, and projection merge into a playful audiovisual experience. Players trigger piano tones by touching colored tiles on a wall, causing the corresponding key to light up—creating immediate visual and auditory feedback.
The project was fully developed in TouchDesigner, with me responsible for the entire process: from concept development to technical execution and on-site implementation. I programmed both the interactive elements and the visual projection.
By making hand movements respond in real time and optimizing the patches, I created a smooth and responsive installation that invites active play and exploration.
The project started with the development of an interactive concept that combines music, movement, and projection. I translated this idea into a technical setup in TouchDesigner. I experimented with camera input, color recognition, and hand-tracking—laying the foundation for the interaction between player and installation.
In this phase, I built the visual system that dynamically projects colored tiles onto the wall and responds to touch. By detecting reductions in incoming light, each hand movement triggers the pressing of a piano key. Every gesture activates both a sound and a visual lighting effect. By tightly linking image and sound, the installation becomes intuitive and immersive for the user.
After the base system was working, I focused on optimizing the installation. By reducing latency and streamlining parts in TouchDesigner, I made the interaction faster and smoother. The result is a responsive, vibrant experience that encourages players to actively engage.
The idea for Piano Tiles was inspired by popular piano tutorials on YouTube. The challenge was always to replicate the complex tones in real life. This sparked the design of an interactive installation that makes piano playing more accessible and engaging.
The entire visual and interactive system of Piano Tiles was built in TouchDesigner. The setup detects hand movements and activates the colored tiles on the wall, providing players with immediate visual and musical feedback.
A projector was used to display the dynamic piano tiles onto the wall. Players wore black gloves, allowing the system to accurately track their movements and making the interaction smoother and more precise.
How interactive projection and sound can subtly influence behavior—transforming stairs into spaces for movement, safety, and play.
What did Utrecht look like during World War II? And how has the city changed over the years? I created an interactive arcade installation that brings the past to life. By using physical buttons on an old map of Utrecht, visitors can activate historical locations and smoothly transition between past and present using a time slider. AI-animated images make history literally melt over the current city, offering a unique perspective on Utrecht’s transformation over the decades. A fusion of technology, history, and design in one immersive experience.
I started with extensive research in the Utrecht archives, selecting historical photographs based on impact and relevance. Using AI upscaling and colorization, I enhanced these images to create a vivid and realistic result.
With Runway ML, I transformed static photos into dynamic videos. I then filmed the present-day locations and combined the footage in After Effects to create seamless transitions between past and present.
Through brainstorming sessions, the idea emerged to present the experience within an arcade cabinet. I built the cabinet in the HKU wood workshop and integrated buttons and a slider for an intuitive user experience.
Using Arduino and TouchDesigner, I programmed the interactive elements. Buttons activated locations, while the slider allowed users to smoothly switch between time periods. User testing helped refine and optimize the experience.
Locations can be selected via illuminated buttons, instantly displaying the corresponding historical scene on the screen.
CoNTENT
Looking for a more immersive experience? Explore the full interactive installation and truly step through time. Visit us and experience the past and present in a unique way!
A mid-WWII photo showing the Maliebaan as a Nazi hotspot. In the center stands Arthur Seyss-Inquart, leader of the NSDAP in the Netherlands, with Anton Mussert beside him. This image is a powerful reminder of the occupation’s impact on Utrecht.
Taken on Liberation Day in 1945, this image shows Canadian tanks rolling through Lange Vliestraat. In the background, the Vroom & Dreesmann (now Bijenkorf) is visible. This moment marked Utrecht’s liberation, symbolizing hope and the war’s end.
A gripping photograph capturing a harsh moment during Utrecht’s liberation. A suspected Nazi collaborator is publicly humiliated by city residents. This image reflects the complex emotions of liberation, from joy to justice, and the harsh realities of war.
This photo captures Canadian soldiers leaving the city with their tanks, while celebrating locals cheer their newfound freedom. It symbolizes both the end of the war and the joy of Utrecht’s citizens as they reclaimed their lives.
A flag with the inscription “V = Victory” hangs from the Conservatory towards the current bowling alley. This flag embodies the euphoria of Liberation Day and optimism for Europe’s future after the victory over Nazi Germany—a powerful symbol of freedom.
.webp)