Algorithm Development

We have created an innovative approach towards making our music. First, we select the top peak frequencies, and manipulate them to allow for proper matching to tuned music notes. Then the frequencies along the x, y, and z axes are matched to notes, the selection of which is limited by our custom algorithm. Then, we play the frequencies in order of initial magnitude before standardization, one chord created from the combination of one note from each axis at a time; this was a direct way of implementing interesting music in MATLAB.

This sets the basis for our future directions, where we hope to temporally synchronize the movement inputs with the music outputs.

Below, these algorithms are explained in greater detail, while visualizing the process through the Macarena dance.

Selecting Peak Frequencies via Filtering in the Frequency Domain

First, we calculate the number of chords needed to create a song that lasts approximately as long as the dance. Each chord is created by using MATLAB’s inverse fast Fourier transform ifft() function, and will return a signal with the same N number of points as there were used to create the original fast Fourier transform (FFT). Therefore, the length of one full chord would be N divided by the sampling rate (Fs), or N / Fs. However, that tends to produce tones that were longer than we wanted. So, when creating our final music, we only use the first fourth of the signal produced by MATLAB’s ifft() function. As a result, the length of a single chord would be 4 * N / Fs. In summary, to create a song approximately the length of the dance (max_time), the number of chords (Nc) would be Nc = floor(4 * N /(Fs)), floor() rounding down to the nearest integer, as one cannot have part of a chord.

For the initial filtering of all 3 axes, a similar pattern is followed: first, we take the FFT of the signal using MATLAB’s fft() function.

A) FFTs of accelerometer data for the x (left), y (middle), and z (right) axes.


We are taking samples (Fs) of our frequencies at 2,000 Hz, as this allows for a frequency range of -1,000 to 1,000 Hz, as the max_frequency = Fs/2. This allows for a broad selection of tones, while avoiding repetition of some higher, harsher tones that some listeners have found to be unpleasant during the development of this product. If we had taken samples of our frequencies in Matlab using the sampling rate of the data collection on our accelerometer (100 Hz), we would have never been able to create diverse music audible to the human ear. This is because music audible to the human ear is between 16 Hz to 20000 Hz. According to the Nyquist Sampling Theorem, frequencies beyond 200 Hz (given the sampling rate of 100 Hz) would be aliased as lower frequencies due to the low sampling rate. There is relatively little music that can be made between 16 to 200Hz on a 12-tone music scale, in comparison to what can be made between 16-20000 Hz [8].

B) FFTs of accelerometer data plotted with the new frequencies corresponding to the sampling rate of 2000 Hz for all 3 axes


Then, all the peak frequencies are found, defined as a local maxima preceded by a value at least 1 unit less than itself.

C) Filtered FFTs showing only the local maximum frequencies for all 3 axes.


The way we find the local maxima is by using an algorithm developed by Eli Billauer [7]. The algorithm loops through each frequency’s magnitude (from the FFT of the movement signal) and compares each one to the next. If one is larger than the other, the larger one is stored in a temporary list. The algorithm keeps going until it processes the last element of the FFT, where it will return a vector of the indexes where the maximum magnitudes were found.

The top Nc positive peak frequencies are selected, and then mirrored horizontally, to avoid clashing phase shifts caused by not perfectly symmetric FFTs.

D) Filtered FFTs showing only the range of highest frequencies selected for music making purposes for all 3 axes.


Then we spread out and shift the frequencies to broaden the range of possible tones, as well as to ensure all of them are in the audible domain. This is done by finding each frequencies’ index’s distance from the 0 frequency and multiplying the distance by the ratio of the highest frequency selected over the highest frequency before aliasing, making the tones cover the max range of frequencies. The frequencies are shifted by 20 Hz, forcing all of the frequencies above this lower limit of audible frequencies. [3].

E) Filtered and Shifted FFTs with a broader range of tones, frequencies all in the audible range, and the frequencies symmetric for all 3 axes.


Lastly, all the magnitudes are standardized, set to 100 unit lengths to enable proper translation of frequency range during audio file writing.

F) Final FFTs before Pitch Tuning Algorithm, where the magnitudes of the frequencies are all set to 100 units for all 3 axes.

Pitch Tuning Algorithm / Chord Selection Process

Pitch Tuning Algorithm

For the purpose of making specific music chords that sound ‘pleasing’ to the human ear, the x axis has been set up to be the ‘base’ or the ‘root’ note. The y and z axes are tuned accordingly to the ‘root’ note and a randomly selected chord based on some aspect of the root note. This selection produces chords that sound pleasing to the human ear according to a study done at Penn State’s Department of Music [5]. On the other hand, this also allows for some flexibility in pitch for the y and z axes, as our project goal is to inspire users to create their own unique sounds from experimentation with movement. Regardless, an increase in acceleration in any axis will result in an increase in frequency in the respective axis of movement. In music terms, this will increase the pitch of the notes being played. Likewise, a decrease in acceleration will result in a decrease in frequency in the respective axes of movement. To add variance to the pitch of the chords, the user should move frequently and move around in all three directions of motion.

To tune each frequency from movement to a respective piano note, the pitch tuning algorithm finds the lowest distance between each frequency of movement and the piano frequencies and replaces the frequency from movement with the respective piano note. The lowest distance can be impacted for the y and z axes depending on the chord selected. When these restrictions apply, the piano frequencies that do not line up with the selected chord (i.e. notes that are not present in that chord) are not considered when finding the lowest distance between the frequencies from movement and the piano frequencies.

For the x axis, the base note is determined by finding the remainder of the index selected for the music note divided by 12 (mod(music_note_index, 12)), and setting all values where the base note is 0 to 12, to ensure compatibility with MATLAB’s indexing. After an initial base note has been selected, the next frequency is selected by first matching the frequency to the closest octave value of the base note, then allowing it to either remain as is, increase by 5 notes, or decrease by 5 notes.

For the y and z axes, the frequencies are matched to the closest value for all octaves of three different notes. The three different notes are determined by the chord selection process below and are based off of the corresponding x frequency (the base note).

The pitch tuning shifts any frequencies higher or lower than the maximum/minimum piano frequencies to frequencies within the range of the piano (7458 Hz and 16 Hz respectively). The algorithm acts similarly to a high and low pass filter in a sense that frequencies outside of this range are shifted, but it is not actually filtering out these frequencies. Our team did not remove these frequencies because these frequencies added more variety to the pitch of the music. By shifting the frequencies to a range audible to the human ear and to a range playable by the piano, they no longer become the objectively bad-sounding or inaudible frequencies that they once were.

If you want to look more at the music theory behind the frequencies of Western music and the pitch tuning algorithm, please read more here.

Chord Selection Process

There are six chords that can be selected at any given time. The process of selecting a chord is randomized so that the generated music does not sound repetitive. This design decision was made only after experimenting with the chord selection being attached to other variables such as the note of the y axis or z axis; we found that when there was little movement around either axis the generated music played the same three notes repeatedly.

Table of All Possible Chords

Based off the chord selected at random and the incoming frequency from the x axis, there are three possible note choices (integers) to select for both the y and z axes. These three integers correspond to the notes needed to produce the selected chord in the 12-tone Western music scale. If you are curious about why these specific numbers are being added to the base note and why we have selected only these 6 chords, please read more about the music theory behind the process here.

Playing Back the Music

The three frequencies played over each time step in the music created by movement during the Macarena dance.

The final music with all three axes combined from the Macarena.

The music along the x axis for the Macarena.

The music along the y axis for the Macarena.

The music along the z axis for the Macarena.

We played the frequencies in the order of decreasing initial magnitude before standardization, one chord created from the combination of one note from each axis at a time. To create this, for each frequency we ran MATLAB’s ifft() algorithm, where the symmetric FFT only had a value of 100 for the sole frequency that was to be converted into music. To make the music higher tempo, only the first quarter of each note being played was strung together into one longer progression of notes. The resulting music from all three axes was then combined in a .wav audio file, with each axis being a different channel.