PlantBand

Designing Shared Musical Experiences Through Plant-Based Interaction for Social Connection

The high-fidelity prototype in action

Situation

As part of an HCI course, our team of six was tasked with designing an intervention for a self-chosen social shared space. We picked the urban park: a place that brings different groups of people physically together, but rarely gets them to actually interact with strangers.

Task

After fly-on-the-wall observations and a failed early concept (an NFC-tag escape room that turned out too abstract), the team landed on music as the mechanic, inspired by the public piano at Utrecht Centraal. My role was building the hardware and embedded system that made this physically work: touch-sensitive plant leaves that play music and light up.

Action

Designed and built the electronics around an ESP32-S3, chosen for its CPU headroom and GPIO count
Used MPR121 capacitive touch sensor boards (8 of 12 channels per plant, one per leaf) wired to copper tape on the leaf edges
Drove individually-addressable NeoPixel LEDs per leaf, daisy-chained so one GPIO pin controls a whole plant
Used an M5Stack Synth Unit (MIDI synth) for sound, with a rotary-encoder + LCD song selector as the UI
Solved signal integrity over distance by running power + I²C over Ethernet cable between the control box and each plant
Wrote the firmware tying it all together (touch detection → light feedback → song-state tracking → MIDI output)

Close-up of PlantBand leaves showing the copper touch-trace patterns and embedded LEDs — Leaf / copper-tape touch points

Early low-fidelity cardboard mockup: a paper leaf and plant next to a cardboard box labeled Turn to select song with two handwritten song names — Early low-fidelity concept: cardboard song-selector mockup

Result

The high-fidelity prototype (2 of 3 plants finished in time) was tested with 9 participants in 3 groups. Average social-connection score (UBC-SSCS) was 5.44/7, and usability (SUS) was 81.9/100, both strongly positive. Participants specifically called out "playing a song together" as what helped them connect, and found the lights/colors inviting.

Reflection

This project let me combine my embedded systems hobby with HCI in a way that's directly testable: it wasn't just "build cool hardware," it was hardware whose only job was to lower the threshold for two strangers to do something together. The Ethernet-as-I²C-bus trick and the LED-density-per-leaf-size lesson are exactly the kind of hands-on debugging I enjoy, but the SUS/UBC-SSCS results are what proved the design decisions (not just the engineering) worked.