Monalisa: "see the sound, hear the image”

 
 


Monalisa is a software platform for Mac OSX, which enable to "see the sound, hear the image”.


It consists from three software: Monalisa Application, Monalisa-Audio Unit, and Monalisa-Image Unit, and an installation: Monalisa “shadow of the sound”


We show the essence of the software and the installation with video.


Monalisa Application is a software works on OSX that transparently treats the image data and the sound data. With this software, people could read the sound data as an image and vice versa. It provides sound and image effects for the data. People could add sound effects to the image data and vice versa.


Monalisa-Audio Unit is an Audio Units* plug-in which bridges the image effects world to the sound effects world and a one of premiere GPU based audio processing software in the world. It wraps Image Units** plug-in as Audio Unit plug-in. With this software, people could use every Image Unit image effects (e.g. motion blur, posterize) as sound effects within Audio Unit host application (e.g. Apple Logic Pro, Apple Garage Band).


Monalisa-Image Unit

Monalisa-Image Unit is an Image Units plug-in which bridges the sound effects world to the image effects world. It wraps Audio Unit plug-in as Image Unit plug-in. With this software, people could use every Audio Unit sound effects (e.g. delay, band pass filter, reverb) as image effects within Image Unit host application (e.g. Apple Motion, Apple Quartz Composer).


Monalisa “shadow of the sound” is an installation, which enables to "see the sound, hear the image". It mainly consists from custom version of Monalisa application and a set of projector, camera, speaker, and microphone situated in a room. In this installation, people see the shadow of the sound of their image. This installation is first premiered at NTT ICC from 2006/6/6 to 2007/3/11 as a part of permanent exhibition.


Here is the trajectory of the installation:

- Take people's photo as an image.

- Generate the image as a sound from the speaker.

- Catch the sound with the microphone.

- Re-project the sound as the image.


As a result of this installation, the sound reflects the optical situation of the room, and the image reflects the room acoustics of the room.



*Audio Units is a system-level plug-in architecture for sound effects on OSX.


**Image Units is a system-level plug-in architecture for image effects on OSX.

 

Members:
Norihisa Nagano
and
Kazuhiro Jo


Norihisa Nagano

Norihisa Nagano is a freelance programmer. Born in 1977, currently based in Oogaki, Gifu. He mainly develops Mac OSX application and web application systems. He started his career for music composition with Max/MSP, then gradually shift the direction into programing with C, Objective-C, and PHP. He develops Monalisa from 2004. From 2007/04, he studies at IAMAS as a master course student.


Kazuhiro Jo
Kazuhiro Jo is an artist and a researcher. Born in 1977, currently based in Tokyo. He graduated from Kyushu Institute of Design, graduate school division of Audio and Visual Communication Studies (MDes). After working at Software Development Lab, IBM Japan, he is doing his research at RCAST, University of Tokyo as a research fellow. His research area is Auditory Interaction Design. He also acts as a co-organizer of dorkbot tokyo(http://dorkbot.org/dorkbottokyo/), a member of The SINE WAVE ORCHESTRA(http://swo.jp/), a member of a performance unit named aeo with eye (Boredoms) and Taeji Sawai(portabole[k]ommunity), and a member of Monalisa project.


Created by

  1. • Norihisa Nagano

  2. • Kazuhiro Jo


Icons are designed by

  1. • Karl D.D. Willis 
    http://www.darcy.co.nz


Supported by

  1. • FY2005 IPA Exploratory Software Project
    (Project Manager: KITANO Hiroaki)

  2. • NTT InterCommunication Center


All software are available at

  1. http://nagano.monalisa-au.org/

Description of the work

Video of the work