To flirt with madness.

The verge of madness.

It is sort of funny, earlier in this week I had an interview with Kurzweil music. Although, the interview went really well, we decided I probably was not the best fit for the position they had at this time. [They need someone who can program closer to the metal then I am used to]. Still, it was a good interview and ironic because of one of the topics we talked about.

Trying to program audio for a system is to flirt with insanity.

The topic came up because we made the observation one cannot simply cannot debug audio. With just about every other form of computing, if something bad is happening, one can freeze to debug a system to figure what is going wrong. But if you freeze a system running audio, the audio stops. So trying to figure out if something wrong with your programming truly becomes a task for a mad scientist. That is what happened this week.

First, I started the week totally excited and really to go. I had all my classes set, the code looked good and… crash. Why? Well, I soon discovered the beautiful sample bank I had created for sound generation was too big. Actually, specifically it has too many sample files in it.

I assumed that every instance I created for Apple’s sampler player AudioUnit, AUSampler, would take care of the files I load into each instance. I was mistaken. Apparently, CoreAudio as an environment has a maximum limit on the number of samples all the instances of AUSampler can have open. This is really weird, because it means the AudioUnits are more closely coupled to the greater audio sub-system then one would be let to believe. After some experiments I discovered that limit seems to be around 250 [probably 256]. The problem was over the summer I crafted a near perfect sample bank, for one of the sounds I need for the project. I was forced to cut my beautiful sound bank of 160 samples, down to just 8 [5% of what it was] so I get enough copies loaded into memory. The new bank does not have the fidelity of the old, but it does the job. That was the first blow.

However, after dealing with that problem, and building a new sampler bank, I ran into a new problem… silence.

For most of the week, I have been tearing my hair out trying to figure out why my new sample bank would not produce sound. It was made it maddening, because of a tiny test audio functions I wrote. The sound would play perfectly with the test function and not for anything else.

Initially, I thought it had to do something with the structure of my code. Specifically, my test functions build, plays, and destroys everything to play a sound all with-in the scope on one function. Since I have to spread the elements for the audio over a couple of classes for the application proper, I thought the way Swift deals with referencing data might be causing the problem. Because sometimes Swift passes data by value and other time by reference. So I thought maybe I was not actually playing audio but just a copy. However, that rabbit hole ended in a dead end. The real culprit was something else.

I noticed one little difference in my test code vs. the project code, which led me to the answer. I am triggering all my audio via MIDI commands to AUSampler. I noticed my test code triggered a sound using a velocity of 1. Compared to the main project code which was using the MIDI max velocity of 127. This led me back to look at the NEW sample bank I had created at the beginning of the week. For some reason AUSampler had mangled the key mapping for the sample bank, and in the end the bank would only respond to a velocity of 15 or below.

OK… I know for the non-MIDI-literate a lot of that sounded like Greek. So let me explain otherwise. I started out by building a Lamborghini. However, after discovering how much insurance was on the Lamborghini, I cannibalize the Lamborghini and built Toyota Yaris from a parts. However, in building the Yaris, something went wrong, and now the Yaris can only drive in first gear.

After making a correct test bank late last night that responds to all the velocity level [so I can drive past first gear], I realized all my code was working, so now I just have to finalized the new bank once more.

Despite the madness there were two silver linings this week. The first, is in rebuilding the sample bank, I had to revisit a Python script I made this summer to create the sound files using SOX. Over the summer, I had so thoroughly documented and robustly wrote that script that I was able to use it to generate new sample files I needed with one minor modifications. SOX, although a great tool, is not the more intuitive command line audio program, especially if you want to do more complex tasks. I think that was first time, I realized I have been able to re-use code I have written in the pass for a current practical task. It was nice to take some solace in that thought.

The second, in trying to chase down reasons the audio would not play, is that it forced me to take a good look at the code I had. Through that, I was able to refactor and simplify some of the code to be able to test and use which I am sure will help going forward.

About ForeverTangent
Currently a Masters of Computer Science Student at the University of Kentucky. Previous Masters of Entertainment Technology from Carnegie Mellon. Before that Graduated from Berklee College of Music. I have worked for Public Radio and the Video Games industry. Most of my interests now are UI and Accessibility Issues for Technology.

Comments are closed.