Generating Real-Time Visual Meaning for Live Indian Drumming
Abstract: In this essay, a system developed to generate visual meaning for live performed music is described. Specifically, this system is calibrated to respond to North Indian classical and folk drumming tradition, using custom designed digital musical interfaces, such as the Electronic Tabla and Electronic Dholak.
A drum, when struck, does not generate its sound as a record of the force applied, but as an artifact of a physical response to that strike within the artistically controlled conditions of its material state. With the design of the North Indian drum controllers, we developed a physical model for digital audio ...view middle of the document...
- Rhythmic visualization of our system used in live performances.
Veldt is an application which was designed from the ground up for the purpose of visual expression and performance. It receives MIDI (Music Instrument Digital Interface) and OSC (Open SoundControl) messages from digital musical interfaces and maps them to a system of reactive events in order to generate live visuals, which are rendered real-time using the OpenGL graphics language. Mappings are flexible: sets of mappings may be arranged and modified during the design and rehearsal process, and triggered by control events during different movements of a performance, and arbitrary text, images, video, and geometric models may be used as source material.
We display a real-time composition of these media sources over geometric elements which are generated and modified according to the parameters of the current mapping. In addition to control events received from the performer, a physical simulation environment is incorporated to allow for a variety of secondary motion effects. This visually (and contextually) rich combination of source material over physically reactive structural elements allows for a response that is dynamically generated and artistically controlled. While the parameters that govern the overall response of the system to the drum controllers may be modified through cues such as MIDI program change messages, Veldt allows an additional visual performer to control the finer aspects of the performance.
Digital Indian Drums
One challenge of this project was retrieving musical data from a performing musician quick enough to make a visual response in real time. There are many methods of generating MIDI or OSC messages from audio data by extracting features from the frequency domain and time domain. These methods proved to be too slow, and inaccurate for real time applications as the number of parameters increased. Hence, we choose to build instruments which could perceive the gestures of the performers, using sensors to capture data in real time, and convert them directly to MIDI or OSC messages. The instruments we will discuss are the Electronic Tabla and Electronic Dholak.
The Electronic Tabla
Tabla are a pair of hand drums traditionally used to accompany North Indian vocal and instrumental music. The silver, larger drum is known as the Bayan. The smaller wooden drum is known as the Dahina. We model the Electronic Tabla (ETabla) on the hand positioning and movements of the many elaborate stroke techniques. We use a set of force sensing resistors to measure the pressure and location when fingers strike the drums. All data is converted to MIDI signals and sent out via a MIDI outport, triggering both sound and graphical feedback (using veldt). For our visual response system, we focused on two strokes performed with the left hand, known as Ka and Ga, to make it easy for the audience to grasp how graphics were being...