hello and greetings from lisbon. things have been busy since we left amesterdam, a couple of weeks ago. i’d like to post the wonderful time at steim. and also some followup on how the workshops relate to my work, and where to now. this post will wander a bit around the wirelessness of it all, about physical wireless controllers and camera invisible interfaces.

steim is the analog resource haven, where a wonderful crew engages all aspects surrounding creation of custom controllers/synths. it has cables, boxes, tables filled with everything useful to electronics all around. and some of the best people handling all of that.

what most caught my attention were wireless controllers being develloped. Michel Waisvisz’s hands are a super example of that. whatt power! in the diy, joysticks, wireless joysticks, and how they can be hacked to almost any kind of analog 5v sensor, how you can boost the signal to provide bigger range using a bit more voltage on the sender, like cell-phone batteries, which is great! joysticks are great ways to pass on data around, and can be very useful for theater/dance, for instance, with camera tracking going on too. they can be invisible triggers in a scene where the camera is not precise enough, or take advantage of a physical situation, like in Joergens’ example, a football wired with accelerometers hacked into a wireless joystick sender.

this gave me a lot of ideas to build wireless controls as instruments that musicians can integrate very well. last week i was programming and performing with parque in festival escrita na paisagem, where i devised the sound->spatializing->lights program on two of three different devices that were played. they were ‘os’ and ‘peça de embalar’ (lullaby piece). the third one is a pendular suspended speaker called ‘pêndulo’.

‘os’ is a piece where 4 musicians play the mirrors suspended, and has microphones attached to each man. these 4(5 with the saxophone) sounds are spatialized in the quad audio setup we had at that piece’s space. there is a graphical video-projection that pinpoints where the sound sources are, and how loud they are. the motion of the spating goes from circular runs to smooth splines traversing space at sound speed, or rapid chance operations.

as interfaces, the macro-parameters were controlled by me in a behringer midi controller and also the computer, but the musicians controlled how fast the sound travelled in the path on some programs, or controlled the light programs and the lights according to each of their sound, on each side of ‘peça de embalar’ through midi pedals, with very very long chords. a wireless joystick can make the same task and even better, you can perform your instrument, and at the same time, control algorithms that use the sound as input and also some accelerometers attached to your hand, for instance. these kinds of ideas spurred at steim. and very likely to develop more as interfaces.

processing live sound is something i have been doing in my installation series ’struct’, where microphones gather site-specific ambiances of galleries or other spaces, and the sounds are continuously recorded into audio buffers that are manipulated by the computer in the first 4 versions, by the sound presence of the user in the 5th version, and by the video presence of people in space in the 6th version ’struct_5′

my interest with controllers now is invisible controllers, mostly camera based tracking, and how one or more people can interact with a digital work. i have also built a different camera approach in 747.3, where a user’s position of arms, and the distance from hand to hand control the heading and the speed of traveling across a digital abstract terrain. here is an image of the object i coded in jitter for this work. one of the main problems/peculiarity of this type of interface is the lack of physical feedback, where the actions that you develop as a user have just a virtual counterpart, and no touch envolved at all. it has its upsides too. quite portable and lightweight. a lot of people enjoyed very much the immersion provided by my piece, and where physically exhausted mostly in the arms. but they relate they forget the real body and immerse themselves in the picture, having really fun runs of 15 to 30 minutes. sometimes it’s also a bit shaky if the conditions are not good. there’s a lot to take in as variable to be controlled in this. the lights have to be very stable, the camera must be blind to visible light, it interferes too much. and finally, the ir lightining must be well made to get good difference images and no shadows interfering. it can also be quite interesting to get different patterns of motion sculpting algorithms of image and sound. like mobs of people or single users specific motion.

future posts will wonder a bit in detail about the topics i mentionned in the works above: creating immersive audio-visual environments, intermedia instruments and camera tracking interaction, all coming around in a new simple work i am developing to be shown as an installation in the lisbon nip workshops.

the nip workshops got a great head start with a heads up on each others work. we started off with building bridges, literally with spaghetti, and metaphorically, grouping and coming up with ideas for new instruments in performance, or what new instruments by means of hacking-programming circuit bending may generate new interfaces. it’s very nice to see that machines are winding everyones brains into a twirl in computation, hacking or building interactive devices to operate with.

we were at watershed in bristol, a great space with rooms to work and present, a nicely crowded bar, some cinema rooms, and a lively open wireless network.

the presentations in the afternoon accounted for our group diversity; very interesting presentations by everyone, relating their projects to the themes of research in creating new engaging works that challenge and dazzle the users.

throughout the day there was a nice atmosphere of opening and sharing experiences. and of how to learn to build from there. how to use cameras, circuits, controllers, algorithms, video, sound, weird joysticks and midi connections, there was a nice buzz in the air.

the next few days were all around programming in zach’s openframeworks project, a great simple powerfull raw level of computation with all this nice libraries being wrapped. the whole thing displays direct through opengl and is very very fast, no other layers other than the code and the libraries. it’s all written in c++, directed at mac and windows platforms with linux coming up. here’s a picture of what i was having fun, feedback and array fun (there’s also a broken graphics card here…)

we were on our way to handy computer vision code, started looking at how to make centroids, camera configurations. mostly exploring code from morning to evening, going through the examples, modifying them, learning how they’re built, creating new stuff from simple examples.