WIP [ITP Blog]

Tangible Interaction - Week 4

February 20, 2019

We were asked to build a MIDI instrument for our second Tangible Interaction assignment. Our project was to provide a tangible interface that could be figured out, manipulated, and played with variation by the user in a repeatable fashion. I was inspired by maracas and set out to build a MIDI version using an accelerometer sensor to detect shaking motions.

maracas

Learning MIDI

My first step in building a MIDI instrument was to learn more about the protocol and build a simple button based instrument. I followed Tom’s MIDI USB and MIDI button instrument tutorial which allowed me to send MIDI commands from an Arduino MKR 1010 to a software synthesizer on my laptop.

midi instrument button

Video demo

Since I’m learning Max in the IDM Sound Studio course, I decided to send the Arduino’s MIDI commands to a Max patch that forwarded it to my laptop’s internal synth. Using a Max patch also allowed me to debug different MIDI commands, parameters, and the sounds of my laptop synth. I constantly referenced the general MIDI standards and MIDI events.

max midi

Accelerometer Tests

Once I had a working microcontroller based MIDI instrument, I then rented an ADXL 335 accelerometer sensor from the ITP shop and built a simple circuit that would calibrate and print to serial the G forces registered by the X, Y, and Z axis inputs. I thought an accelerometer would be an appropriate sensor after the feedback Stefan received on his hourglass like kitchen timer project to look into an accelerometer or gyroscope to register the turning over motion/event.

adxl 335

accel plotter

Video demo

Working with the ADXL 335 also taught me about using an AREF pin as an “analog reference” voltage for higher accuracy. One thing to note is that for the MKR board, a slightly different constant (AR_EXTERNAL) needs to be referenced when activating the external AREF.

Accelerating MIDI

The first way I combined the MIDI circuit with the accelerometer circuit was to trigger a MIDI “note on” command when the G forces on the X axis exceeded 0 (video demo). I then experimented the levels of possible musical interactions by adding a button that would change the MIDI instrument sound (video demo), giving the single X axis two musical dimensions.

I continued to expand on my instrument’s interactivity by incorporating the recorded calibration axis values I had gathered from the sample accelerometer circuit. When the acceleration exceeded an absolute threshold (which would happen when the circuit is moved quickly, i.e. shaked by the user), the MIDI commands to “note on” and “note off” would be sent for the given axis affected (video demo of shaking, video demo with button option). Since the acceleration value could be variable based on how fast the user moved the instrument, I decided to also utilize that value by mapping it proportionally to the MIDI note velocity value. This created the effect that the faster the shake, the louder the note.

Fabrication

I returned to the ITP junk shelf to find an appropriate enclosure. There was a mailing tube that was large enough to fit the electronics, but small and round enough to be gripped by one hand. It also reminded me of a rainstick — an instrument I typically associate with percussive ensembles.

junk shelf tube

The tube’s bottom was made of metal, so to avoid conductive issues and provide padding for the electronics I filled the tube with soft fabric scraps. I also ran the tube through the band saw to shorten its length.

tube bottom

tube fabric filling

I then purposely drilled the hole for the instrument’s panel mount button in the tube’s middle to indicate the instrument should be held with one hand and the button pressed with the thumb. The photos below are from me testing different drill sizes on the tube’s unused other half.

tube button hole

tube button fit

Indicating Sound

During my testing, it became apparent that the MIDI note velocity cue wasn’t significant enough to indicate a difference between shaking the instrument faster or slower. So I incorporated an LED thats brightness was proportional to the acceleration too.

final

Here is a video demo of Luming testing the finished instrument and the source code used.

Future Thoughts

Some thoughts I had if I continue to iterate on this idea:

  • Make the instrument wireless to allow maximum movement and shaking of the instrument — I experimented with Wifi communication on the MKR, but struggled to get it to work with a MIDI network session on my laptop (even tried using the Arduino-AppleMIDI-Library to no success)
  • Use an LED light strip instead of a single LED for a more explicit indication of volume effect (had issues running FastLED on the MKR, even after following this thread on editing a header file in the library)
  • Experiment with other MIDI commands to trigger when shaking in a certain axis (e.g. Z axis shaking triggers pitch bends) and user test the new interactivity to see what people find more intuitive

Adrian Bautista

A perpetual work in progress blog documentating my NYU ITP projects. Words are my own.