LAC 2015 - Paper: An Update on the Development of OpenMixer

2015·3.1 MB·English

by  Elliot Kermit-Canfield

An Update on the Development of OpenMixer
Elliot KERMIT-CANFIELD and Fernando LOPEZ-LEZCANO
CCRMA (Center for Computer Research in Music and Acoustics),
Stanford University
{kermit|nando}@ccrma.stanford.edu
Abstract was tailored to the Listening Room studio at
This paper serves to update the community on the CCRMA. The control hardware is comprised
development of OpenMixer, an open source, mul- of standard PC computer components housed
tichannel routing platform designed for CCRMA. in a fan-less case and off-the-shelf RME audio
Serving as the replacement for a digital mixer, soundcards, plusexternalSolidStageLogicAl-
OpenMixerprovidesroutingcontrol,Ambisonicsde- phaLink MADI AX and Aphex 141 boxes for
coders, digital room correction, and level metering
A/D and D/A conversion. The control soft-
to the CCRMA Listening Room’s 22.4 channel, full
ware runs on Gnu/Linux and is a collection of
sphere speaker array.
free,opensourcesoftwareprogramscoordinated
Keywords from a master control program written in the
SuperCollider programming language. Inputs
Multichannel Diffusion, Spatialization, Ambisonics,
to the system include 3 ADAT connections to
Open Source Audio Software, 3D Audio
andfromtheListeningRoomLinuxworkstation
1 Introduction (aregularCCRMALinuxcomputerthatcanbe
used for music or sound work and has direct ac-
The Listening Room at the Center for
cess to all speakers), 2 additional ADAT I/Os
Computer Research in Music and Acoustics
for external digital sources, 16 line level ana-
(CCRMA)isaversatilestudiospaceservingthe
log inputs, 8 microphone preamplifier inputs, 8
CCRMA community. Featuring a 22.4 channel
analoginputsforamediaplayer, and4ethernet
spherical speaker array and a 32 speaker wave
jacks connected to a separate network interface
fieldsynthesissystem, theListeningRoomisan
(whichcanalsoprovideInternetaccess)thaten-
important facility for research, music composi-
able computers which have NetJack easy access
tion, and 3D sound diffusion among other uses.
to all speakers and features of the system.
As conventional mixers are limited in config-
urability, channel count, and usability, we have
At the time the previous paper was written,
implemented a software routing system—called
the Listening Room was equipped with only 16
OpenMixer—to control the primary speaker
speakers in a 4 + 8 + 4 configuration, which
system.
could either be addressed independently or be
Although conceived in 2006, the implemen-
fedthesignalsofanAmbisonicsdecoder(second
tation of OpenMixer began in 2009. Since
order horizontal, first order vertical) integrated
then, the capabilities of the Listening Room
into the system. Two Behringer USB control
developed—there are more speakers, faster
surfaces(BCF200andBCR200)providedasim-
computers, and more users. To meet the de-
ple and easy-to-use user interface.
mands, OpenMixer has been under continuous
development. This paper serves as an update
toitscurrentstatusandextendsLopez-Lezcano
and Sadural’s 2010 LAC paper [1]. 1.2 Changes
1.1 Original Implementation
OpenMixer is our hardware and software solu- The following sections outline most of the
tion to the problem of routing, mixing and con- changestoOpenMixersince2009, bothinhard-
trolling the diffusion of multiple multichannel wareandintheunderlyingcontrolsoftwareand
sources over a large number of speakers, and functionality.
2 Hardware Upgrades
2.1 Speakers
The number of speakers in the Listening Room
has grown over time. The last major upgrade
happened in March 2011, when the number of
main speakers was changed from 16 to 22 (23 if
we count an additional “center channel” which
is only used for more accurate 5.1 or 7.1 play-
back). The spatial configuration was changed
from 4 + 8 + 4 to 1 + 6 + 8 + 6 + 1,
Figure 3: 7 speakers below the grid floor
which enabled us to cover the sphere more uni-
formly and thus decode full periphonic third or-
der Ambisonics with high accuracy (see figs. 1 at 2.83GHz, an EP45-DS3R Gigabyte mother-
to 3). In addition, four subwoofers were added, board, and 8G of RAM to a newer quad core
whichrequiredaddinganadditionaleightchan- i7-3770K CPU running at 3.50GHz and using
nel ADAT D/A converter to the equipment a DZ77GA-70K Intel motherboard with 32G of
rack. RAM. At the same time we moved from the
old Fedora 14 platform to Fedora 20 (new real
time (RT) patched kernel and newer versions of
all packages). An updated hardware signal-flow
chart can be seen in fig. 4.
The upgrade was anything but boring. Al-
though much faster, the upgraded computer re-
fused to work with the PCI audio cards used
by the old hardware. We were using two RME
cards in tandem—one RME Hammerfall DSP
MADI PCI that interfaced with the main SSL
A/D D/A box and an RME 9652 PCI card for
additional ADAT ports. The upgraded moth-
erboard included two legacy PCI slots for that
purpose, but we were never able to have both
cards working at the same time. The second
Figure 1: The Listening Room
card would fail to get interrupts delivered to it.
This was difficult to debug and we spent count-
less hours switching cards around. We eventu-
ally had to move to newer PCI express cards
with equivalent functionality (an RME HDSPe
MADI and an RME RayDAT PCIe).
Even then, the technique of aggregating both
cards into a composite ALSA device using
.asoundrc did not work anymore. The newer
kernel, ALSA drivers, and hardware could not
get the card periods aligned, even though the
soundcardsaresyncedusingwordclock. JACK
Figure 2: 7 elevated speakers
wouldimmediatelystartgeneratingxrunswhen
started on the composite device—each card by
2.2 Computer and Peripherals itself would work fine. To get around this prob-
One of CCRMA’s fan-less Linux machines is at lem we tried running JACK on only one of the
the heart of the Listening Room—OpenMixer cards and connecting the second card to the
system. As we needed more processing power system. We did this through either JACK’s
to support some of the planned expansions of audioadapter internal client or alsa_in and
the OpenMixer system, we migrated the hard- alsa_out clients. In both cases, the DSP load
warefromanoldquadcoreQ9550CPUrunning was too high even for the upgraded hardware
able through custom UGens and plugins and
handles multichannel audio, MIDI, and OSC in
a native and intuitive fashion.1 SuperCollider
itself is really two programs—an interpreted
language (sclang) controlling a separate syn-
thesis server (scsynth) running as a separate
unix process.
In this new version of OpenMixer, we
switched the synthesis server to SuperNova, an
alternative server written by Tim Blechmann
which supports automatic parallelization of the
DSP load across multiple cores [4]. This simpli-
fiedthesoftwareconsiderably, asbeforethis, we
were performing load balancing across cores by
instantiating several instances of scsynth and
controlling which server was chosen for particu-
lar tasks by hand.
When the OpenMixer computer boots, Su-
perCollider runs automatically as a boot ser-
vice and starts all other ancillary software. The
Figure 4: System hardware diagram maintaskofSuperColliderisroutingaudiofrom
multiple sources (Linux workstation, ADAT,
analog, BluRay, mic preamps, and networks
platform.
sources) to the speaker array, as well at to
We finally used Fons Adriaensen’s Zita-
the Linux workstation and networks sources for
ajbridge to add the second card as a separate
recording. This represents a complex many-to-
JACK client [2]. As in the previous solutions,
many routing relationship through the use of
thisperformsunnecessarysamplingrateconver-
software buses (see fig. 5). In addition to rout-
sion of the second card inputs and outputs, but
ing straight from a channel to some number
it was the only way to get the system to work
of speakers (each with independent gain con-
reliably and the additional load was acceptable
trol), OpenMixer also supports decoding of up
(between 9 and 10% of a core each for 64 in-
to third order Ambisonics audio streams. An-
put and output channels). Another twist to the
other important task that SuperCollider per-
story is that we could not get the MADI card
forms is as a task manager for subsidiary pro-
to run as the master card when using the Ray-
grams such as JACK and Jconvolver. If any of
DAT as a secondary audio card. We had to run
OpenMixer’s auxiliary processes dies, the sys-
the RayDAT as the master, which increased the
tem restarts them automatically.2
channel count to sample-rate conversion as well
as the load. All these sound driver peculiari- 3.2 System software
ties need serious debugging, but we just did not
Together with the hardware changes, we up-
have the time to do that.
graded the operating system to Fedora 20 and
Forcaseslikethisoneitwouldbenicetohave
a new RT patched kernel [5]. To get the best
an option in Zita-ajbridge to not do sampling
real-time performance from the new system, we
rate conversion at all, as the cards run in sync
needed to perform a few new tweaks to its con-
through word clock.
figuration.
We found that the Intel i7-3770K pro-
3 Software Upgrades
cessor together with the new kernel (cur-
3.1 SuperCollider and Supernova rently 3.14.x with the latest RT patch)
used the new intel_pstate frequency
The majority of OpenMixer runs inside Super-
scaling driver instead of the conventional
Collider, a programming environment specifi-
cally designed for audio applications [3]. In ad-
1We could only imagine the horrors of programming
dition to being well supported by a large devel-
OpenMixer in C or C++!
oper/user community, SuperCollider is extend- 2Except in very bad situations...
the cost of increased thread scheduling latency.
We don’t know how much latency HT actually
adds, but it seems that it is enough in our cur-
rent mix of tasks and threads to negate any ad-
vantage in raw processing power HT might of-
fer [6–8].
The start-up scripts that boot OpenMixer
have also changed. The boot activation of the
software now happens through a static systemd
service. The service executes a script that
Figure 5: Simplified OpenMixer software rout- tweaks root level access configuration settings
ing diagram and then starts the main start-up script as the
“openmixer” user. This in turn starts a private
XserverforfutureGUIextensionsandtransfers
governor-based scaling cpupower software.
controltosclang(theSuperColliderlanguage).
The intel_pstate driver can be controlled
The script also re-starts sclang if for some rea-
through variables in the /sys filesystem
son sclang dies.
(/sys/devices/system/cpu/intel_pstate).
The following incantation was used to tell the 3.3 User Interface
driver to try to run all cores at the maximum The user interface in the previous version of
speed all the time: OpenMixer was limited to two Behringer con-
trol surfaces to access most common settings.
echo 100 > /sys/devices/system/cpu/ This was a deliberate choice to keep the inter-
intel_pstate/min_perf_pct face simple and mode-less, where every button,
fader, and knob had only one clearly labeled
We also disabled the thermald daemon which
function. Most of the interface has not changed
we normally run in all our Linux workstations,
but some new functions have been added (see
asitalsotweaksthespeedoftheprocessorcores
fig. 6). For example, the master level control
if the thermal load exceeds preset limits. We
knobs also double as level meters for each bank
knowwhattheloadontheprocessoris, sothere
of 8 channels (you can select the function of the
isnoriskofthermaloverloads. Inbothcases,we
knobs),thereisa“DRCon/off”buttonthatcan
want the core speed to be pegged to the max-
be used to disable the correction filters on each
imum available. It looks like the scheduler is
speaker, we now have a monitor option (with
not entirely aware of the current speed of each
optional pre-fader level) that routes any input
core and can migrate a computationally heavy
back to the Linux workstation, we implemented
task from a core running at full speed to one
a speaker test function to quickly make sure all
that is idling (and running at a very low clock
speakers are operational and also a software re-
frequency). The intel_pstate driver will of
set that reinitializes OpenMixer.
course adjust the speed of the slow core, but
not fast enough to avoid an xrun in some cases.
We also had to disable hyper-threading (HT)
intheBIOS.Thisdisablestheadditional“fake”
cores and brings down the total number from
eight to four (but those four are real cores—we
willuseacheaperInteli5processorswhichlacks
hyper-threadingforfuturedeployments). With-
out these changes, we experienced occasional
xrunsinJACK.HTcreatestwologicalcoresout
of each physical core by storing extra state in-
formation that enables both logical cores to ap-
pear to be independent processors. The poten-
tialimprovementinperformanceisonlyrealized Figure 6: User interface
by highly threaded applications and tops out at
about 20% in ideal conditions. This comes at We have also employed SuperCollider’s QT
GUI objects to build a graphic user interface so ber of external programs used, JACK graph
that a monitor connected to the control com- complexity, and context switches.
puter can display relevant information. For
3.6 Digital Room Correction
now, we have only implemented input and out-
put VU level meters, but this change opens the From the start, OpenMixer calibrates all speak-
doortofutureupgradesthatshouldmakeOpen- ers for uniform sound pressure level and delay
Mixer even simpler to use, specially if we use a at the center of the room. While the new lo-
video screen with touch control. cation of the speakers in the studio covers the
OpenMixer is internally controlled through sphere with better resolution, they are not re-
OSC messages. We have been therefore exper- ally suspended in a free field condition. In
imenting with applications like TouchOSC on particular, the speakers located under the grid
tablets or smart phones to be able to control floor (below the listener) have a different tonal
OpenMixer remotely through simple apps in- quality due to the physical construction of the
stalled in user’s equipment [9]. “pit” in which they are located. Furthermore,
not all speakers are the same exact model (al-
3.4 Subwoofer Processing
though they all share the same high frequency
The addition of subwoofers to the Listen- drivers). We have Mackie HR824s at ear level
ing Room required us to implement proper andsmallerMackieHR624sfortheelevatedand
crossovers in software. We used LinkwitzRiley lowered speakers. The result of this led to in-
4th order crossovers implemented as SuperCol- correct and sometimes confusing rendering, es-
lider UGens that are part of the instruments pecially when decoding full sphere Ambisonics
that do software mixing and routing within (manifested through a tendency of sounds to be
OpenMixer. “pulled” towards the ceiling).
InthenewversionofOpenMixer,wehaveim-
3.5 Ambisonics Decoding with ADT
plemented digital room and speaker correction
and ATK
using Denis Sbragion’s DRC software package,
First through third order Ambisonics decoders
as described by Joern Nettingsmeier [15]. By
are naively supported by OpenMixer. In the
recording the impulse responses of each speaker
previous version of OpenMixer, Ambisonics de-
at the listening position, inverse FIR filters can
coders were provided by running ambdec_cli
be calculated to even out small differences in
as a subprocess of the master SuperCollider
speaker impulse response due to the speakers
program, with decoder matrices tuned to our
themselves and their location in the room.3
speaker arrangement kindly supplied by Fons
This, coupled with the very dry acoustics of
Adriansen. A stereo UHJ decoder was also pro-
the studio itself makes for an accurate and even
vided by running Jconvolver as another subpro-
reproduction of sound over the whole sphere
cesswiththesuppliedUHJpresetconfiguration.
around the listening position. The only trade
In the new version of OpenMixer, we have
off is an increase of approximately 10 millisec-
switched to using decoders calculated by the
onds in the latency of the system. We have
Ambisonics Decoder Toolbox (ADT), written
added an on/off switch to be able to bypass the
by Aaron Heller and Eric Benjamin [10–12].
DRC generated correction filters and access the
ADT consists of Matlab/GNU Octave scripts
speakers directly if necessary.
that calculate Ambisonics decoders though a
Jconvolver, written by Fons Adriaensen, is
variety of techniques. In critical listening
used as an external subprocess of the main Su-
tests, the decoders have proved to perform very
perCollider program and provides an efficient,
closely to the previous hand-tuned ones. In ad-
real-time convolution engine that performs the
ditiontogeneratingAmbdecconfigurationfiles,
filtering [17].
ADT writes the decoders as Faust programs
whichcanthenbecompiledtocreatenativeSu- 3.7 Adding HDMI inputs
perCollider UGens [13]. For stereo UHJ decod-
We recently had the need to allow users to
ing, we have switched to the Ambisonic Toolkit
connect HDMI audio based equipment (specif-
(ATK)UGens, writtenprimarilybyJosephAn-
ically game consoles—for research, of course—
derson[14]. WehavebeenabletofoldAmbison-
ics decoding entirely into SuperCollider which 3Weused20secondlogarithmicchirps,recordedand
minimizes the complexity of the code, the num- analyzed with Fons Adriaensen’s Aliki [16].
and also laptops). This adds another way to Onesuchpiece, arecordingmixedthroughAm-
connect equipment to the system and can be bisonics,isrenderedbothinfullthirdorderAm-
used to play back 5.1 or 7.1 content (easier in bisonics and stereo UHJ in a time-synced way
those cases than using NetJack or analog in- so the “immersive-ness” of the decoders can be
terfaces). We bought an Outlaw Model 975 audified and evaluated.
surround processor, a low cost unit that pro-
vides both HDMI switching and decoding of 4 Future Work
themostcommoncompressedsurroundformats
Although OpenMixer was originally written for
(DolbyDigital, DTS,DolbyTrueHD,andDTS-
the CCRMA Listening Room, we are working
HD Master Audio, etc). The analog outputs of
to parameterize the scripts so that it is more
the unit were connected to the “player” analog portable.4 Once appropriately parameterized,
interface, and the existing Oppo BluRay player
weenvisionOpenMixerasausefultoolforother
was connected to one of the Outlaw’s HDMI in- peoplecontrollingmultichannelspeakerarrays.5
puts.
We are in the process of implementing Open-
The only gripe is that the unit cannot (yet) Mixer in our small concert hall, the Stage, that
be controlled from OpenMixer itself. It would is equipped with a 16.8 channel 3D speaker sys-
be nice to be able to control everything from tem. OpenMixer will make it possible to easily
the OpenMixer control surfaces so that the user move pieces from the Listening Room environ-
does not need to know which HDMI input is ment to the concert hall. This is especially true
which and how the surround processor is con- intheAmbisonicsdomain,asnewdecodertech-
trolled. We could use an infrared remote con- nologiesimplementedinADTmakeitrelatively
trol, or inquire if the RS232 interface included easy to design effective 3D decoders for dome
in the unit can be used for control in addition speaker configurations such as the one we have
to firmware upgrades. on the Stage. CCRMA concerts frequently fea-
ture large speaker arrays (up to 24.8) which are
3.8 Demo Mode
controlled with similar systems. Making Open-
OpenMixer makes the Listening Room easily Mixer a general solution would simplify these
configurable, however, it requires some effort large scale productions.
between walking into the studio and hearing Much work remains to be done in the soft-
sound. In the past, demonstrations of the Lis- wareitself,inparticular,tousethenewX-based
tening Room’s sound system—which are quite GUI interface and to control the whole system
frequent—typically included hunting for saved through OSC from tablets and smart phones.
Ardour sessions to recall and some amount In the Listening Room itself, we have plans
of guesswork to find projects spread across to add more subwoofers (for a total of 8) to
CCRMA’s filesystem. To simplify this process, increase both the fidelity and localization reso-
we implemented a demo mode that highlights lution of low frequencies.
the capabilities of the system in an easy to use As the technology gets faster and the number
way. We used a Korg nanoKONTROL2 as a of speakers grow, OpenMixer will most likely
controlsurfacethatispatcheddirectlytoOpen- remain a work-in-progress.
Mixer. This controller has eight channel strips We originally intended to use JackTrip as
(three buttons, a knob and a fader) and trans- an additional input source for remote perfor-
port control buttons. We have pre-rendered mances, but the feature did not see much de-
an assortment of multi-channel works for the mand and it was impossible to make JackTrip
Listening Room’s speaker configuration that work (without changes to the source code) as a
can be easily triggered through the nanoKON- reliable subprocess to the SuperCollider code.
TROL2. We plan to use the currently unimplemented
With minimal effort, someone can listen to a JackTripinputstohouseFonsAdriansen’sZita-
curated set of mixes of great variety. We have njbridge (network-JACK) JACK clients, thus
included selections that show off different types
of 3D diffusion (e.g., quad, 5.1, octophonic, 4Naturally, this is a challenging task as speaker con-
figuration, sound cards, input sources, etc. are different
third order periphonic Ambisonics, etc.) as well
for each system.
asmixingstyle(e.g.,“concerthall-esque,”“fully 5MoreinformationaboutOpenMixercanbefoundat
immersive,” “academic electroacoustic,” etc.). https://ccrma.stanford.edu/software/openmixer.
providing a way to connect to the Listening computing clusters,” Linux Clusters Insti-
Room from remote locations [18]. tute Conference, 2002.
[8] Calomel.org, “Network tuning and per-
5 Conclusions
formance: a simple guide to enhancing
OpenMixer is certainly not the only solution
network speeds.” https://calomel.org/
for controlling 3D speaker arrays at CCRMA,
network_performance.html.
but it has fulfilled its purpose to make the Lis-
tening Room configurable for many different [9] Hexler.net, “Touchosc: Modular
uses. The latest inclusion of DRC-based im- osc and midi control surface for
pulse response calibration has significantly im- iphone/ipod touch/ipad.” http:
proved the quality of sound diffusion in the Lis- //hexler.net/software/touchosc.
tening Room, and is being considered for other
[10] A.HellerandE.Benjamin, “Ambisonicde-
listening spaces at CCRMA.
coder toolkit.” http://www.ai.sri.com/
ajh/ambisonics/.
6 Acknowledgments
[11] A. Heller, E. Benjamin, and R. Lee, “A
The success of OpenMixer is largely due to
toolkit for the design of ambisonic de-
the support of the CCRMA community. Many
coders,” Proceedings of the Linux Audio
thanks to Chris Chafe and Jonathan Berger for
Conference 2012, 2012.
embracing the Listening Room in their classes
and projects. OpenMixer is built on free, open [12] A. Heller and E. Benjamin, “The ambison-
source software and would never have existed icsdecodertoolbox: Extensionsforpartial-
without all who contributed to those projects. coverage loudspeaker arrays,” Proceedings
of the Linux Audio Conference 2014, 2014.
References
[13] Y. Orlarey, “Faust: the quick path from
[1] F. Lopez-Lezcano and J. Sadural, “Open-
ideas to efficient dsp.” http://faust.
mixer: a routing mixer for multichannel
grame.fr/.
studios,” Linux Audio Conference, 2010.
[14] J. Anderson, “The ambisonic toolkit: tools
[2] F. Adriaensen, “Zita-ajbridge, alsa-
for soundfield-kernel composition.” http:
jack and jack-alsa resampler.”
//www.ambisonictoolkit.net/.
http://kokkinizita.linuxaudio.
org/linuxaudio/. [15] D. Sbragion, “Drc: Digital room cor-
rection.” http://drc-fir.sourceforge.
[3] J. McCartney, “Supercollider: real-
net/.
time audio synthesis and algorithmic
composition.” http://supercollider. [16] F. Adriaensen, “Aliki, impulse re-
sourceforge.net/. sponse measurement tool.” http:
//kokkinizita.linuxaudio.org/
[4] T. Blechmann, “Supernova: a multipro-
linuxaudio/.
cessor aware real-time audio synthesis en-
gine for supercollider,” Master’s thesis, TU [17] F. Adriaensen, “Jconvolver, a convo-
Wien, 2011. lution engine.” http://kokkinizita.
linuxaudio.org/linuxaudio/.
[5] Various, “Real-time linux patches.”
[18] F. Adriaensen, “Zita-njbridge, network-
https://rt.wiki.kernel.org/index.
php/Main_Page/. jack and jack-alsa resampler.”
http://kokkinizita.linuxaudio.
[6] A. Valles, “Performance insights to
org/linuxaudio/.
intel hyper-threading technology.”
https://software.intel.com/en-us/
articles/performance-insights-to-
intel-hyper-threading-technology.
[7] T. Leng, R. Ali, J. Hsieh, V. Mashayekhi,
and R. Rooholamini, “An empirical study
of hyper-threading in high performance