Audio and Video

The URL of an audio or video file can be attached to an event. (In the 1.2 version of Quotidian, the implementation of video is experimental and may not work well in many circumstances.)

The QViewer uses the GStreamer 0.10 System to play audio and video. There are hundreds of formats for media files and GStreamer has a plugin architecture that supports many of the widely used formats. Many formats require license fees to use legally. Quotidian only distributes GStreamer for the Mac and Windows platforms and only distributes plugins that are unemcumbered by license or patent restrictions. Most Linux users will already have GStreamer on their machines. If not, they can install packages created for their Linux distribution. Users on any platform are free to build their own versions of GStreamer 0.10 and replace the ones supplied with Quotidian.

Supported Audio Formats

 

  • WAV
    Wav is a standard format for audio on personal computers. The audio data is usually uncompressed and the files can be quite large.
  • Ogg Vorbis
    Ogg vorbis is an audio compression format. It is roughly equivalent to mp3 but is completely free, open and unpatented.
  • FLAC
    FLAC stands for Free Lossless Audio Codec. Because it is lossless, the compressed audio files in this format are of higher quality than with the mp3 and ogg vorbis formats and also much have much larger files sizes.
  • MP3
    Mp3 is the most common compressed audio format on personal computers and on digital audio players. It is patented and a license is required to sell or distribute decoders and encoders for this format. Quotidian does not distribute mp3 decoders. On Windows and Macintosh computers, Quotidian uses services provided by the operating system to play mp3 files. For Linux, a fully licensed decoder is provided free of charge by Fluendo (http://www.fluendo.com/shop/product/fluendo-mp3-decoder/). The user can download the decoder in the form of a Gstreamer plugin and add it to the directory that contains the other GStreamer plugins.

Quotidian also provides some support for audio playlists in the pls and m3u formats. These are often used in streaming audio applications. Playlists should work if the entries in a playlist use one of the supported audio formats.

Supported Video Formats

  • Theora
    Theora is a video compression format. It uses the same Ogg container and Vorbis sound encoding as Ogg Vorbis audio files. "It is completely free, open and unpatented."
  • Webm
    The development of this format is sponsored by Google. A WebM file consists of VP8 video and Vorbis audio streams, in a container based on Matroska. The software is distributed under a license that grants all users "a worldwide, non-exclusive, no-charge, royalty-free patent license". This is a fairly new standard but it is also supported in current versions of the Firefox and Google Chrome browsers.

Sound Modes

When audio files are defined in events, they are given one of three play modes:

  • once
    The sound will play once while the display time is between the start and end times of that event. Moving the display time out and back into the event's time range will play the sound again.
  • repeat
    The sound will play repeatedly while the display time is between the start and end times of that event.
  • real-time
    The sound will play only when the QViewer is in real-time mode. In this mode the sound is mapped to the event starting at the beginning of both. The event usually has the same duration as the audio file. If the display time is 30 seconds from the start of the event, the sound heard will be 30 seconds from the start of the audio file. This mode is not defined for audio files without a fixed duration such as audio from streaming sources.

The event associated with the sound does not need to be visible for the sound to play. Events can be defined without a color and without a label. The category to which the event is attached to must be drawable, that is, the data-hiding mechanisms attached to its category must not have completely hidden a category's events if the events could be drawn.

Spatial Sound

Each sound is defined as part of an event. The QViewer's audio system will attempt to mix the left and right stereo volume to mimic the position of the event on the display area. As the event associated with the audio is moved to the left of the display area, the source of the audio should appear to move to the left as well. A maximum of ten audio files can be active at the same time, but in practice this number may be limited by data transfer rates.

Spatial sound can be turned off by setting the enableSpatialSound variable to false with the edit preferences command in the options menu. The default value is true.

Audio Rolloff

There is one aspect of data hiding specifically related to audio. The audio volume can diminish as the category is moved away from the center point of the display area. Like most of the other aspects of category hiding, rolloff is usually not an abrupt cutoff. Rolloff can be given a distance where the volume starts to diminish and an amount that the volume decreases as the edge of the category moves from the center of the display area.

 

The soundSpace demo shows a number of events in different categories. Each has an attached audio clip. The sounds begin to play when the time is moved between the start and end times of the events. As the user moves the viewpoint around the events, the audible sound from each event changes. The rolloff is set to rapidly decrease the volume of an audio clip as the border of an event moves away from the center of the display. The audible sound of an event extends partly into each immediate neighbor event. In the example in the image above, the rain sound is nearest the center of the screen and is playing loudest. The thunder sound is mixed in at a slightly lower volume. The sounds in the other events are far enough from the center of the screen to not be heard.

Audio Delay

There is an initial cost to starting a audio file. This can slow down rapid movement across a large number of events with audio files even when there is no interest in hearing the first instants of the sounds. This is smoothed out by adding a time delay between when an event is "entered" and when the sound starts to play. "Entered" refers to both a movement of the display time between the event's start time and end time and a movement of the center of the display to within the rolloff range of the event's category. The time delay factor can be changed with the soundDelay variable in the edit preferences window.

The default value for soundDelay is 500 milliseconds (a half second).

Enable/Disable Audio

Sound rolloff and spatial sound depend on the relation of the event carrying the sound to the center of the display area. With multiple QViewer windows, the center of the display area will usually be different in different windows. Only one window can have sound enabled at a time. The play sound command selects the window used to calculate audio placement and volume. Choosing this command in different windows may abruptly change the sound. When no window has play sounds selected, no audio is played.

Building and Replacing GStreamer

Expert users are free to download and build their own versions of GStreamer. This is not a trivial task and requires either building or downloading binary versions of many third-party code packages. Information about GStreamer and links to current source code can be found at http://gstreamer.freedesktop.org.

Pre-built GStreamer code is available from http://gstreamer.com.

GStreamer uses several environment variables to find its executables and plugins. Quotidian sets these variables to its own GStreamer distribution when starting on Windows and Macintosh. If you wish to replace the distributed GStreamer with another one, you can set the following variables.

QGST_PLUGIN_PATH replaces GST_PLUGIN_PATH
QGST_LOCATION sets the location of the main GStreamer directory in PATH or LD_LIBRARY_PATH
QGST_REGISTRY replaces GST_REGISTRY
The GST_PLUGIN_SYSTEM_PATH is always turned off. Plugin directories should be explicitly enumerated in QGST_PLUGIN_PATH.

© 2011 Quotidian Incorporated