LAC 2015 - Paper: AVTK - the UI Toolkit behind OpenAV Software

2015·0.22 MB·English

by  Harry van Haaren

AVTK - the UI Toolkit behind OpenAV Software
Harry VAN HAAREN
OpenAV Productions,
Co. Clare,
Ireland.
[email protected]
Abstract and many different solutions exist. Many so-
lutions are platform dependant, ie: they only
AVTK[1]isasmalllightweightuserinterfacetoolkit
designed for building custom graphical user inter- work on Linux, Mac OsX or Windows.
faces interfaces. It was conceived particularly to PUGL[3] is a cross platform library which
build LV2 plugin GUIs, however it can be equally “supports embedding and is suitable for use in
useful in standalone programs. plugins”. AVTKusesPUGLtoembedintohost
Focusing on user experience, AVTK promotes
program’s windows, and it can create a stan-
ease of use for novices yet affords power-users the
dalone window too if embedding isn’t desired.
most efficient interactivity as possible.
The author feels this is particularly important
2.2 Modal Popup Windows
in live-performance software, where user-experience
and creativity are in close proximity. Many general purpose toolkits have the func-
tionality to create modal popup windows. This
Keywords type of window can cause problems when em-
User Experience, User Interface, LV2 Plugins. bedding a plugin UI in a host window.
The first problem is that the popup is set
1 Introduction to be “above” its parent window, however the
AVTK is a C++ user interface library. It is parent window is generally already set to be
designedspecificallyforcreatingLV2plugins[2], “above” the host program. This can cause the
but can be used to build normal user interfaces popup window to be below the parent window,
too. OpenAVcreatedAVTKoutoftheneedfor however modal popup windows don’t allow the
a flexible and powerful way of creating custom user to interact with the parent. The user is
user interfaces for audio plugins. force to manually bring the popup window to
In the background section, common issues the front, and interact with it. This has a neg-
when building user interfaces for plugins are ative impact on the user experience (UX).
presented. The implementation of the AVTK The second problem is one of stalling the UI
user interface library is then shown, and code thread of the host program. When a modal
samples are provided, providing an example of popup dialog is created, many user interface
how to build a minimal interface. Finally we toolkits wait for the user to interact with it,
discuss how the common problems presented in before continuing the execution. This stalls the
the background are solved in this implementa- host programs UI thread, as that thread cre-
tion. ated the dialog. The end result is that as the
popup window is shown, the host programs UI
2 Background is frozen. Again, a negative impact on UX.
Whencreatinganembeddableuserinterfacefor
2.3 User Experience
a plugin, issues arise that do not apply to writ-
ing normal user interfaces. These issues arise When using a plugin user interface, user experi-
from the interaction between the user interface ence is of very high importance. In the context
of the host program and plugin. ofAVTK,theuserismostlikelycreatingmusic,
orinvolvedinacreativeprocessofsomedescrip-
2.1 Embedding tion. ToensurethebestUXforbothnoviceand
Embedding is the process of showing a user in- power users, the normal user interaction con-
terface created in one toolkit inside the window cepts are augmented in AVTK. Care is taken to
of another toolkit. This is a complex problem ensure that these augmented interaction possi-
bilities do not cause confusion to novice users. The value() functions set and get the value
Transparently augmenting the interaction of the widget. A callback function can be pro-
with a widget provides power users with better vided to be notified of activity on a particular
UX, and ultimately in the user achieving their widget instance.
goal more efficiently. The scroll wheel on the
3.3 Minimal UI
mouse, and modifiers keys are used in order to
The code in listing 2 shows a minimal AVTK
augment the interface for efficency.
user interface with a single button. Notice that
3 The Implementation we override the widgetValueCB()function, and
that the callback function of any child widgets
The implementation of AVTK draws from ex-
is automatically routed to the UI instance.
perience gained while developing custom inter-
faces using other toolkits.
Listing 2: Minimal Demo UI
The main window is treated as one large can-
class DemoUI : public Avtk::UI
vas, and widgets are drawn into this canvas. As
{
widgets can be transparent, creating layered in- public:
terfaces becomes easy. DemoUI(PuglNativeWindow parent = 0) :
Avtk::UI( 400, 240, parent )
3.1 Dependencies {
new Avtk::Button( this,
During the design stage cross-platform libraries
50, 20, 300, 200, "Button" );
were chosen, in order to ensure portability.
}
Cairo[4] is used for all drawing routines, while
PUGL provides access the window and input void widgetValueCB(Avtk::Widget* wid)
events. PicoJSON[5] is used for parsing JSON, {
printf("Widget %s with value %f\n",
while tinydir[6] provides access to the filesys-
wid->label(), wid->value() );
tem.
}
};
3.2 Widget Class
The Widget class is the core of AVTK. The int main()
Widget class has virtual functions that a devel- {
return DemoUI().run();
oper can override to customize behaviour. The
}
following is an excerpt of the Widget class:
3.4 Custom Widget Creation
Listing 1: Widget Class
In order to customize a widget one simply de-
class Widget
{ rives from the Widget base class, and overrides
public: the draw() method. The Cairo library is used
Widget( Avtk::UI* ui,
to draw widgets, and a cairo t* context is
int x,
passed into the draw() function. Calling the
int y,
int w, desired Cairo functions will draw the widget.
int h, The value() function of the Widget base
string label ); class can be called to get the current value of
the widget - allowing easy drawing of the wid-
// draw and handle events
get in its current state.
virtual void draw(cairo_t* cr);
virtual int handle(PuglEvent* e); The handle() function can be overriden in
case the custom widget requires non-default
// set value on widget
user input handling.
float value();
void value( float v ); 3.5 Modal Widgets
Modalpopupsareimplementedasapopupwid-
// change notification callback
void (*callback)(Widget*, void*); get, insteadofitsownwindowinAVTK.Thisis
void* callbackUD; to keep UX consistent, and avoid the pitfalls of
};
modal windows as described in the background
Themostimportantfunctionisdraw(),which section.
paints the widget to screen. The handle() As the calling thread should not be stalled
function deals with user input using the cross- whilethepopupisshown,AVTKtreatsapopup
platform PuglEvent abstraction. widget like a normal widget. The exception is
Figure 1: The Fabla2.0 interface, powered by AVTK. Edited for print.
event handling, which allows the modal wid- The following sections discuss where the inter-
gettodisableinteractionwiththeotherwidgets action between user and interface has been aug-
while it is shown. mented.
Figure 2 shows two user interfaces one of
4.1 Right Click and Default Values
which has a modal dialog shown. Note the dia-
log is positioned close to the cursor for ease of User input from the mouse buttons is leveraged
interacting with it. in almost every widget in order to allow effi-
cient resetting of controls. Any Widget that
3.6 Theming
uses its value() is enabled with a right-click
The theme engine in AVTK is geared towards feature to reset to the default value. Calling
providing a variety of themes to a widget, and defaultValue() sets a new default value for
being able to change them on the fly. Themes the widget.
are loaded from JSON files at runtime, allowing Right-click to reset to default value can
easy modification. be disabled, using the Widget::rClickMode()
Theme files provide colours for “use-cases”. function.
Some examples are background, foreground,
4.2 Groups and Scroll events
background-dark etc. This approach allows us-
ing the same theme file on any widget, and the Groups can be used to provide radio-button
widget adapts the colours in the theme to what style selection of a particular widget in the
is being drawn. This lightweight approach to group (see Fig 2. for widget examples).
themeing allows fast prototyping, and scales to This is particularly useful in scrollable areas,
having many different themes available for each which are commonly used in applications to se-
widget. lect a particular option from a range of pre-
Fig. 1 shows an interface, and various wid- determined options.
gets themed to show the user the effect of the Using input from the scroll wheel is an intu-
widget in question. In this particular screen- itive mapping - but there is a conflict when a
shot changing the “Bank” changes the primary Group is in a Scroll widget as the Scroll wid-
colour of the interface, indicating the change to get uses the scroll event. In order to achieve
the user. an optimal workflow, a Ctrl+Scroll action is
added to a Group allowing the user to navigate
4 Special Features
group.
ThissectiondescribesspecialfeaturesofAVTK. The efficiency of navigating a widget list
Theinteractionbetweenthepoweruserandthe is improved for power users familiar with the
interface is where the design of AVTK flour- Ctrl+Scrollhotkey, whileconsistencyismain-
ishes. Many widgets afford operation with tained in how widgets behave regardless of if
hotkeys, right-clicks and drag-n-drop areas. they are in a scrollable area or not.
Figure 2: Showcase of AVTK widgets, a popup Dialog “stripes out” the background.
4.3 Filenames AVTK serializes the input events from the
user to a JSON file, which is uploaded to the
When the user is searching for a particular au-
developer. They then replay the events, auto-
dio file in a list of similar files, the filenames
matically mimicing the users input.
often have an identical start causing long file-
It should be noted that the developer must
names. This causes a workflow where the user
have the same version of the software as the
must scroll horizontally and vertically in order
useraspixelco-ordinatesarestoredintheevent
to select the desired file.
stream. It follows that if the user interface is
In an attempt to solve the workflow issue as
re-arranged, the users actions may no longer
above, the AVTK file-browser includes an op-
achieve the same result.
tion to hide the common start of a filename.
And example to illustrate this is as follows:
5 Conclusion
// Actual Filenames AVTK is a small lightweight user interface
samplepack_kick_heavy.wav toolkit, targeting developers who wish to build
samplepack_kick_click.wav custom user interfaces. It solves common issues
samplepack_snare_snappy.wav other toolkits have when embedding as a plugin
UI by utilizing a more appropriate design for
// AVTK Filenames the use-case.
kick_heavy It has some special features geared towards
kick_click power-users,andhasathemeenginethatallows
snare_snappy developers create prototypes quickly. An event
recording and playback mechanism is included
The example above shows three filenames with to aid finding issues users are having with the
thecommonprefixsamplepack followedbythe software in question.
information user requires, followed by a filetype
6 Future Work
extension.
In this example, the common prefix OpenAV intends to continue using AVTK to
samplepack has been removed, and the build interfaces for plugins and standalone soft-
extension is removed for readability (It is noted ware. Currently the Fabla 2.0 sampler is the
that many general purpose toolkits already only complex project using AVTK.
hide the file extension). Future work includes expanding the available
widgets as necessary for multi-media centric
4.4 Testing of User Interfaces
software, and testing on all major platforms.
When an issue is found in a program by a user,
often the first step a developer takes is to at- 7 References
tempt to reproduce the issue. The user is asked [1] http://openavproductions.com/avtk
to describe what steps will reproduce the issue, [2] http://lv2plug.in
and the developer mimics them in order to find [3] http://drobilla.net/software/pugl
the cause of the issue. [4] http://cairographics.org
To improve the workflow in finding UI issues, [5] http://github.com/kazuho/picojson
userinputisrecorded, andthenreplayedonthe [6] http://github.com/cxong/tinydir
developer’s computer.