This post is about ENSAL residency (Sept 1-7, 2013) and the two weeks prior
After the initial CYBORGAME rehearsals, I spent several weeks tightening the software and fixing various bugs. I also had an opportunity to spend a week working with David Chanel (of Theorize Crew) who was working at the time in the EPFL LTS2 lab. Among other things they are responsible for a system which tracks commuter patterns in the Lausanne Train station by stitching together the data from more than 60 depth cameras.
After the initial CYBORGAME rehearsals, I spent several weeks tightening the software and fixing various bugs. I also had an opportunity to spend a week working with David Chanel (of Theorize Crew) who was working at the time in the EPFL LTS2 lab. Among other things they are responsible for a system which tracks commuter patterns in the Lausanne Train station by stitching together the data from more than 60 depth cameras.
David and I discussed the overall tracking problem that I've been tackling (covering a stage-sized area by stitching depth data) and he convinced me to change my approach.
Until CYBORGAME I had been planning to push all of the point cloud data from each remote Kinect through the network. In fact I did this (and it works pretty well, especially when compressed) but the compression introduces a bit of time overhead and more importantly limits the capability of your average network to 4-6 simultaneous point clouds (if you are lucky and assuming a closed, wired gigabit network).
The Lausanne train station system is far less precise than what I need for stage applications, but it collates the data from many more (approximately 60) ASUS Xtions with little overhead.
The approach is to offload the scene processing to embedded computers (or in my case, to the Mac Minis) which are connected to each Kinect. Software on these machines is responsible for collecting the point cloud data, but also analyzing it and running background subtraction, cleaning the noise, and ultimately sending a simple dataset which consists only of points of interest.
As a starting point, David and I spent a week working on TSPS, which is an openFrameworks based system capable of doing visual tracking and scene analysis at fairly high frame rate. It also has the advantage of being (theoretically) cross-platform.
 |
| TSPS showing one of our modifications - the addition of depth camera support, clipping and projection |
TSPS is under active development, and I'm sure this is outdated as I type it, but it was built for and around optical tracking (IE webcams). It does an excellent job of this, but for my purposes it was not robust enough and lacked a lot of the support for the depth cameras (Kinect and ASUS) that we are using.
David and I spent the week building out TSPS to include the features we thought we needed for the tracking problem. Following this, I accepted an invitation for a one-week residency at École Nationale Supérieure d’Architecture de Lyon (ENSAL) to work with a team on three installations they had proposed inside the main ENSAL building. For this project there were six of us (a mix of developers, musicians and visual effects folks). We split into groups of two and presented the result at the end of the week.
In addition to testing the TSPS based system, I wanted to begin shifting course away from purely technical development and towards the architectural ideas which sent me on this course in the first place. I have in mind some notions of architectural memory
To that end I used the week to develop ReOSC, an OSC logger and playback tool that we can use in conjunction with a tracking system to record the movement in a space over time. The idea is to collect data that comprises the "memory" of a space. On a purely instrumental level this can be used to determine usage patterns, etc, but I am more interested in the conceptual: Invisible lines of desire and the ways we traverse space. Can a building dream? Can a space mourn the loss of an occupant?
 |
| ReOSC: Tool for recording and playback of OSC data |
ENSAL has a basilica layout, with a large central corridor ending in a multistory circular rotunda that houses the program offices. This main "street" connects all the labs and classrooms on the ground floor and provides access to the exists and stairways to the upper level. It can be observed from a second story balcony and, during the day, is brightly lit by a glass ceiling that runs the length of the building.
 |
| The main hallway of the ENSAL building where the residency took place |
In addition to developing ReOSC, for the residency I created a very simple projection/interaction system which tracked people moving down the hallway and provided them with a colorful contrail describing their trajectory. This simple interaction allowed me the time and space to work on the tracking/recording system and activated the space with an element of pleasant surprise. In some cases individuals traversing the hallway ignored the installation completely (still providing a small light show to incidental observers). In other cases, individuals spent a good deal of time playing with the light, making shapes and experimenting with interactions between each other. My favorite interaction was an older man who I was later told was a retired Swiss judge. He spent 20 or 30 minutes dancing with the system.
Although it provided a good base to work from, after this week I concluded that TSPS is not up to the job after all, due mostly to the user interface code. In theory I could re-write this, but it seems unwise to do battle against the framework at its core. Instead we are decomposing the problem and working on platform-native code that should be more stable for stage applications.
