Table Of ContentELECTRONIC
MULTIMEDIA PUBLISHING
Enabling Technologies
and Authoring Issues
edited by
Fillia Makedon
Dartmouth College
Samuel A. Rebelsky
Grinnell College
A Special Issue of
MULTIMEDIA TOOLS AND APPLICATIONS
An International Journal
Volume 6, No.2 (1998)
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
MULTIMEDIA TOOLS AND APPLICATIONS
An International Journal
Volume 6, No. 2, March 1998
Special Issue: Electronic Multimedia Publishing: Enabling Technologies
and Authoring Issues
Guest Editors: Fillia Makedon and Samuel A. Rebelsky
Guest Editorial .......................... Fillia Makedon and Samuel A. Rebelsky 5
MediaWeaver-A Distributed Media Authoring System for Networked Scholarly
Workspaces ......................................................... Sha X in Wei 9
ASML: Automatic Site Markup Language ... Charles B. Owen and Fillia Makedon 25
AsT#-Towards Modality-Independent Electronic Documents ....... T.V. Raman 53
Structural Queries in Electronic Corpora ........... Daniela Rus and James Allan 65
Obstacles in Web Multimedia Publishing: Bringing Conference Proceedings
On-Line . . . . . . . . . . ........ Peter A. Gloor, Fillia Makedon and Oliver Van Ligten 83
Resource-Limited Hyper-Reproductions: Electronically Reproducing and Extend-
ing Lectures ................ James Ford, Fillia Makedon and Samuel A. Rebelsky 93
Library of Congress Cataloging-in-Publication Data
A C.I.P. Catalogue record for this book is available
from the Library of Congress.
ISBN 978-1-4757-8271-4 ISBN 978-0-585-34906-0 (eBook)
DOI 10.1007/978-0-585-34906-0
Copyright © 1998 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 1998
Softcoverreprint ofthe hardcover1st edition 1998
All rights reserved. No part of this publication may be reproduced, stored in a
retrieval system or transmitted in any form or by any means, mechanical, photo
copying, recording, or otherwise, without the prior written permission of the
publisher, Springer Science+Business Media, LLC.
Printedon acid-free paper.
•~•, Multimedia Tools and Applications 6, 93-96 (1998)
© 1998 Kluwer Academic Publishers.
Guest Editorial
Advances in computing technology-particularly in processors, storage, displays, and
networking-have changed the requirements for publishing and introduced the concept
of electronic publishing, in which documents are created, modified, and presented electron
ically using mechanisms like CD-ROM, computer networks, or both. Electronic publishing
has many similarities to traditional publishing, as authors still create manuscripts that are
edited, designed, and published. Electronic publishing also differs from traditional pub
lishing in many ways: it can include time-based media (such as audio and video), it allows
more dynamic updates (the creation of a new edition of a printed text is difficult while elec
tronic texts may be updated easily and frequently), it has richer environments for using and
manipulating texts published electronically, and it follows a somewhat different authoring
strategy designed to accommodate not just the creation of content, but also the choice of
what media to use to present that content.
Web-based publishing is a form of electronic publishing that uses the World Wide Web
as a distribution vehicle. It permits an array of novel applications, such as "interactive
authoring" in which readers play a direct role in the construction and experience of a book,
not only through powerful tools they use to search and manipulate the content, but also
through feedback and annotation that can expand the book for other readers. In that sense,
readers can play a key role in the evolution of an electronic publication.
Electronic multimedia publishing also permits enhanced data access or retrieval. To
replace facilities of traditional publishing, such as using pencil, highlighters, bookmarks,
margin notes and physical structure to use an authored document, electronic publishing can
include a set of tools that empower the reader.
Multimedia (as opposed to "monomedia") publishing combines a variety of forms of
data, not just text and images, but also video, audio, animations, simulations, and more.
Multimedia authoring means considering not just content, but how to present that content.
Just as one might have a picture in a story book of the traditional format, we now may have a
picture that is a video clip, an animation, a simulation, something else that a creative author
imagines, or a combination thereof. In addition, vast repositories of digital information or
applications may either become connected to the authoring process directly with links, or be
made a part of a multimedia "document" interface. An authored multimedia document offers
"information windows" to worlds of data that can be accessed and processed , unl' form or
another. Electronic publishing may include interfacing with or building virtual r useums,
digital libraries, virtual theaters, document archives, on-line banking, and much more.
Given this fusion of modalities in multimedia publishing, it becomes harder to distinguish
where the function of "publishing" ends and where a program simulation begins. One
criterion that distinguishes electronic publishing from, say, programming a video arcade
game is that authoring is involved to convey a concept, thought or idea, rather than simply
to create an interactive environment without content. Thus, content is the defining criterion
94 MAKEDON AND REBELSKY
that distinguishes one publication from another, rather than the technology of delivery. It
would, of course, be ideal to have infinite portability on the fly and be able to read electronic
multimedia publications seamlessly across platforms and systems and across media, but this
is still not possible in the current world of electronic publishing.
In this issue of the Journal of Multimedia Tools and Applications, we have brought
together a number of papers that describe tools and strategies for electronic publishing.
These papers have grown out of discussions continued from the DAGS95 (Dartmouth
Institute for Advanced Graduate Studies) Conference on Electronic Publishing and the
Information Superhighway that took place in Boston in June of 1995. While the DAGS95
conference covered a broader range of issues, including low-level supporting technologies,
legal implications, and the effect on the publishing industry, in this issue we have chosen
to focus on representative papers that cover enabling technologies for the basic steps of
electronic publishing. We see five steps in this process: ( 1) the initial step of idea generation,
which is not a technical matter and is not covered in this issue; (2) content acquisition and
authoring; (3) system and user interface desing; (4) information or document retrieval; and
(5) dissemination, maintenance, and extension.
Multimedia electronic publishing is traditionally based on an authoring system. Author
ing systems are primarily used in step 2, content acquisition and authoring, but may also
affect the interface design, retrieval mechanisms, and the final stage of dissemination and ex
tension. This issue includes papers on two fairly different authoring systems or capabilities:
a software framework (MediaWeaver) for composing distributed multimedia applications
supported by a variety of platforms, and a WWW-based system, called ASML. In his paper,
Xin Wei describes MediaW eaver, a rich environment for managing and producing a wide
variety of media over variable network services and standards. MediaW eaver emphasizes
collaborative creation and use of multimedia documents, an essential distinction from tradi
tional publishing. In their paper, Owen et al. describe ASML, the Automatic Site Markup
Language, a hybrid markup and programming language designed for creating uniform and
reconfigurable multimedia web sites with minimal or no programming effort on the part of
the user. ASML is particularly appropriate for non-technical users who nonetheless want
more capabilities than are available from most authoring tools, and promises powerful,
site-level search capabilities, global editing, and automatic image markup.
System and user interface design must incorporate user access and interaction. Just as
electronic multimedia publishing offers alternative ways of authoring with multiple media, it
also offers the user multiple ways of"multi-reading" a particular electronic document. In his
paper on audio browsing, Raman describes ASTER (Audio System for Technical Readings),
a system that audio-formats electronic texts and thus makes this information accessible to
the visually disabled. This paper outlines some of the key ideas for understanding the use
of multiple modalities in publishing and for understanding how to represent media so that
they can be presented in multiple ways.
Information retrieval tools for finding and extracting information from electronic pub
lications are an essential enabling technology for making multimedia publications usable
and useful. In their paper, Rus and Allan describe a retrieval engine for text and image
documents that is based on document layout. They describe a methodology for enabling
the user to construct structural hyperlinks automatically, thus enabling access to electronic
6
GUEST EDITORIAL 95
publications by querying in search of some text or a figure. Other issues in retrieval include
queries based on multiple media, rather than a single medium (primarily text or image).
A multimedia publication involves more than tools and authoring systems. It also requires
efficient processes for dissemination, maintenance and update. Gloor et al. address this
issue by describing trade-offs and design decisions for authoring multimedia conference
proceedings for the World Wide Web. Their system is among the first of its kind to include
audio, images and interactive search features and remains one of the best. In a similar vein,
Ford et al. describe a process for multimedia authoring appropriate to those with limited
resources, although in some aspects inapplicable to multimedia authoring in general.
These papers touch upon key issues in multimedia publishing. However, there are many
other current and future issues that are of great interest to both the publishing and multi
media communities. These include enabling technologies for compression of documents
and accessing compressed documents; high-level abstraction of video publications with still
images (representative frame techniques); the ability to authenticate the origin or ownership
of publications with watermarking of images, video, audio documents; new retrieval and
organization methods; programming languages for manipulating and creating multimedia
publications; tools for the automated processing of multimedia data that go into a publi
cation; and better understanding of the underlying social, legal, and economic issues of
electronic publishing. We expect to revisit some of these issues in a future issue.
Fillia Makedon
Professor of Computer Science
Dartmouth College
Hanover, NH
Samuel A. Rebelsky
Department of Mathematics and Computer Science
Grinnell College
Grinnell, lA
Fillia Makedon is a Professor of Computer Science at Dartmouth College since 1991. Before that she was
Associate and Assistant Professor at the University of Texas at Dallas and at the Illinois Institute of Technology
in Chicago. She received her Ph.D. in Computer Science from Northwestern University in 1982. She is Director
and Founder of the Dartmouth Institute for Advanced Graduate Studies in Parallel Computation (DAGS Institute),
which she co-founded in 1992. Professor Makedon is also Director of the DEVLAB (The Dartmouth Experimental
Visualization Laboratory), which focuses on proving basic research tools and new algorithms for multimedia
7
96 MAKEDON AND REBELSKY
systems and applications. She is currently supervising five Ph.D. students and her interests are in the areas of
multimedia information retrieval, audio and video analysis, electronic publishing and multimedia interfaces for
digital library applications. She is author of numerous research articles, and recipient of many awards. She is the
mother of three children, Basil, Dana and Calliope.
Samuel A. Rebelsky is an assistant professor of Computer Science at Grinnell College. Prior to coming to
Grinnell, he was on the faculty of Dartmouth College and Assistant Director of the Dartmouth Experimental
Visualization Laboratory. He received his Ph.D. from the University of Chicago in 1993, where his research under
Michael O'Donnell was on program communication and programming languages. At Dartmouth, his research
involved the development and analysis of hypermedia systems and the application of hypermedia to education.
His current research emphasizes languages and systems for hypermedia manipulation.
8
Ill..•,
Multimedia Tools and Applications 6, 97-111 (1998)
' © 1998 Kluwer Academic Publishers.
MediaW eaver-A Distributed Media Authoring
System for Networked Scholarly Workspaces
SHAXINWEI [email protected]
Sweet Ha/14/5, Stat!fiml University. Stanji1rd. CA 94305
Abstract. We describe Media Weaver-a software framework for composing distributed media in the context
of university research and instruction. Authors compose networked media, software tools and mediastreams, and
can freely annotate media by media of any form using schema of their own design. Faculty and student authors
compose distributed media using common Macintosh, World Wide Web and NeXTSTEP applications, supported
by services from UNIX workstations.
The MediaW eaver system mediates between network multimedia services and interface kits with which novice
programmers and non-programmers may easily create radically different interactive views into shared mediabases.
The network services include search engine abstractions. filters, relational modeling frameworks.
MediaW eaver has supported collaborative projects in history, drama. music, art, anthropology, environmental
studies, and other fields since 1993. Applications range from traditional relational text databases and indexed
HTML WWW sites to course readers. research archives, journals and seminar spaces.
Keywords: multimedia. distributed databases. humanities computing
1. Introduction
A major challenge facing designers of networked computing environments today is to
fashion scholarly workspaces which are simultaneously coherent, easily reconfigurable,
efficiently expressive-small gestures go a long way, and above all, worth using.
In this paper, we describe MediaW eaver, a system that has streamlined the composition
of arbitrary renderable media, mediastreams and applications in diverse models and narra
tive structures. MediaW eaver is designed to support the construction of models of human
systems which are both conceptually rich and data rich. It also mediates between coherent,
customizable interfaces and an open set of network services, such as database engines,
WWW servers, full text and image search engines [ 13] 1, and media conversion facilities.
And it is designed for open sets of media that will change over time.
Our context is humanities computing [ 12], which significantly stretches the envelope
of networking technology, multimedia, intelligent search systems, and human-computer
interface design. Software technology paradigms now run the gamut from verb-object
tools ("set the color of the selected word to red") to document processing, intersubjective
computing and urban design [3]. We take a perspective situated somewhere between in
tersubjective computing and urban design. Our method has been to have designers and
programmers work intimately with the faculty and student researcher/authors who use the
evolving systems [6]. In fact, Media Weaver was conceived in the beginning as a framework
to accelerate our own multimedia designers' work in creating rich complexes of media
98 SHA
supported by relational data models. But it was natural to extend the role of the designer to
include authors who were experts in fields outside computer engineering.
2. The problems
After about five years of making interactive multimedia applications, we took stock of our
work process to see where the bottlenecks were, and also what were the greatest defects in
the interactive titles produced for scholarly applications.
• Media were scattered all over the network. It was becoming hard to keep inventory using
ad hoc databases.
• Researchers significantly changed their conceptual models over the course of a project,
so that custom data structures had to be re-written.
• User interfaces had to be constantly re-designed in concert with graphics artists, pro
grammers and researchers, using unpredictably varied media. New interface constructs
such as help sprites and custom gestures which did not fit pre-fabricated window-menu
button widgets had to constantly invented.
• Finished titles were often locked into a videodisc or piece of software (e.g., Director or
Supercard stack), and put out of reach of re-purposors.
• Finished titles had thin media content/hard content boundaries-users quickly hit the
boundaries of what was recorded on a CD ROM or videodisc.
• Conceptual models were often too simplistic to be taken seriously by any but the most
novice students. We wanted environments which could support research level work as
well as introductory classes. (In general, software which was designed specifically for a
given class or lesson was often too rigid or shallow.)
• Hypertext/media graph topologies were either navigable but too sparse to sustain a
viewer's interest, or rich but too dense to be comprehended. Hypertext links are fragile,
difficult to author or manage, and hard to map.
• We could not easily support multi-author and multi-player discourse networks.
The MediaW eaver was designed to address all of these problems. Its various frameworks
were designed to be used by faculty and student authors and by designers of multimedia sim
ulations. It was designed explicitly to be usable by members of academic disciplines outside
computer science and engineering. And it had to leverage tiny application programming
resources.
We started with two prototype projects in 1993-1994: a history of Renaissance
(Elizabethan) theater, and a study of high technology in the Silicon Valley. The first was
chosen from a pool of faculty projects which required some management of art images
and associate music or text on the network. The second presented the challenge of deal
ing with a significant, changing body of structured text in a complex, evolving research
project. In addition, we wanted to lay the foundation for general relational modeling of
human systems as such data became available in the course of the research. In both cases,
we could not assume a fixed interface or conceptual model. Indeed, the only surety was
change.
10
MEDIA WEAVER-A DISTRIBUTED MEDIA AUTHORING SYSTEM 99
This genealogy strongly influenced the design principles which we will outline in the
following section.
Since then we have continued with the SiliconBase [9], as the Silicon Valley History
project is called, and have added several other communities and mediabases, including, for
example: a prototype for an archive of electro-acoustic music; a Chicana/o artists database
[15]; a clearinghouse of international conservation information (7]; a history of education
since Greco-Roman times [4]; and a structural engineering database2.
3. Design principles and corollaries
3.1. Make it immediately useful
Bread and butter reasons, but also participatory design principles suggested that we should
let composers start working right away with their own media, conduct seminars and write
articles using our system instead of waiting for the Holy Grail. To enable significant scholarly
work, whatever we built had to exchange data transparently with commercial applications
and databases, and inter-operate transparently with distributed services. Authors were
encouraged to use whatever commercial editors they already had on their personal comput~Crs
(Macintosh or Windows )3. Our frameworks synthesize commercial, public and custom
software. Our authors work in a heterogeneous network where UNIX and Macintosh
clients see a common filesystem, and can apply user tools from Macintosh, UNIX (Sun,
SGI, NeXTSTEP) to shared mediabases.
3.2. Factor, factor, factor
The architecture reflects a separation between (1) persistent storage in the filesystem (e.g.,
ASCII or AIFF blob bytes) and in databases (e.g., blob metadata in Sybase tables); (2) model
(e.g., hypermedia topological structure, bibliography); and (3) presentation/interaction
(e.g., WWW/Mosaic document, Hypercard simulation, custom disposable apps). By de
coupling models from media, we can sidestep the question of data ownership and allow
complex research models to be constructed on existing corpora or proxy media4.
Since MediaW eaver stores topological information in databases, it can generate HTML
documents dynamically rather than keep source media in HTML files-a simple version of
dynamic documents. Factorization gives us the option of interposing even more expressive
and nuanced means of forming constellations media or mediastreams on-the-fly.
3.3. Maintain user interface metaphor neutrality
We wish to allow multiple views on shared media, which means that rather than build
ing a single interface application or layout protocol (a Ia HTML forms), we provide an
API supporting multiple, concurrent, and most importantly, reconfigurable interfaces. The
MediaW eaver does not assume that views must look like word-processors. Word-processor
like document viewers like MS Word or Mosaic present essentially a unidimensional rebus,
11
