nfirvine.comwiki

Nothing explains something better than an analogy:

Essentially, instead of a UI emitting static sounds when clicking a button, this Idea proposes that some sort of a realtime synthesizer run in the background and emit variable waveforms based on various factors. For a trivial example, clicking at point (x,y) would emit a sound in stereo panned to x/screen_width.

Rationale

I feel sound is an underutilised cue in HCI. Sounds emitted by the desktop are generally annoying, so it's easy to understand why it hasn't been so popular. However, in an environment where the desktop is 3D (like inside a cube), having programs emitting their sound in a 3D space would add to the metaphor.

Candidates

• Csound realtime: Csound can be adjusted in realtime. I haven't looked into Csound much. Looks a little beyond my knowledge, but seems to have more than enough power and historical usage in synthesis.
• Python Csound module: Ubuntu package "python-csound"; "import csnd"
• Embedding python in Csound: Csound has some opcodes that can execute python code and get data from it.
• Pulseaudio doesn't appear to have Python support (!). Gstreamer?

Prior Art

• Using 3D sound in HCI