Boards Interactive Magazine – Walkthrough
Questions:
How did this project come about? What was the initial brief from ‘boards magazine?
We were contacted by Nexus Productions who had been given the brief to develop an augmented reality cover for the Boards March issue.
The brief was really open. We knew it would be called the innovation issue which would be focusing on inspirational, non-industry projects, so we thought it would be a great chance to develop something wild and playful for the cover.
How did you come up with your concept of Rise and Fall?
Before we started the project we produced a series of quick experiments to test the tracking of the magazine and discover the types of interaction it allowed. Some of those experiments played with the idea of using the magazine as a tool for navigating a 3D world. We really liked this approach as it allowed us to work at a different resolution and scale than just the camera input. It also seemed quite different from other Augmented Reality projects we had seen. We then started to explore different ways the magazine tracking could influence a dynamically unfolding story. We hooked up the orientation of the magazine to control the gravity of a dynamic world and realized it could be a lot of fun to try and tell a story about opposite forces where rising or falling through the world changed the narrative and the interactions you experience.
Once you settled on the idea, what was your process of testing out both the usability of the interactive aspect and also developing the look of the piece?
Well it was sort of a crazy process of experimentation and refinement! At this stage we were developing small ‘test apps’ that would test potential elements for the final story. The oil and water node is a good example of this. It starts out as its own app and we then see if it feels playful to interact with, if it illustrates a concept and how it fits into the ‘world’ of Rise and Fall. A lot of these ‘test apps’ don’t make it into the final software, but by testing the interaction with the tracking we really get a feel for how well it might work. We initially had the magazine controlling more aspects of the world but we found that this needed to be simplified to the most core interactions as the interface and the content were battling for attention. We made the interaction mainly about the very direct action of rotating the magazine We found this worked a lot better for the user experience, as well as really emphasizing the principle theme of Rise and Fall. The visual style of the piece evolved alongside the development, but the core idea was to have a subtly to the details as you moved between the two contrasting worlds. The visual style of Rise is light and airy, while fall feels somewhat a mix of night and the deep sea. We tried to weave these qualities into the character design and narrative in a deliberately ambiguous way.
Were their any technical challenges that posed a major obstacle to your vision of the piece? Anything you wanted to, but just couldn’t include?
I think the biggest challenge was that the software needed to run fast on many different types of computers. Usually for the installation work we do we have complete control over the hardware which allows us to really push things from a graphical and tracking perspective. A lot of the visuals for Rise and Fall are dynamically generated so we were constantly battling with trying do a lot visually but still keep the app running fast at a solid frame rate. We also have the computer vision running at the same time ( tracking the magazine cover ) so there is quite a lot going on all at once! As a result we had to leave out some of the really heavy programmatic animations and also graphics shaders ( post processing effects ) which could have added some extra visual flair but tend to be very hardware specific. As with most of our work our ideas our always bigger than what we can realistically implement. We still have a big list of crazy things we want to add, as well doing more inter-node interaction, where the nodes react and respond to one another. We would be really excited to work these things into a larger scale installation version though, maybe with 3D glasses too?
Any pleasant surprises?
The workflow we developed for the project was a really nice surprise. We have our own workflow for working on installations but there was a lot of new tools and techniques which we developed for this project. For Rise and Fall we were working with so many assets that it forced us to come up with clever ways to manage and load them into the software. We developed asset managers for type, sound, image sequences and colors, so that we were never loading anything more than once. We also developed a color system that used names rather than values so that it if we decided to change the color palette it was a simple matter of swapping out an xml file. Another nice surprise was seeing some of the early programmatic experiments mature into a seamless blend of designed and programmatic elements, like the hairy beasts whose eyes track your movement! The behind the scenes mode ( which is activated when you show the back of the magazine ) was a lot of fun to work on too as it is giving people a peek at the forces at work, the tracking and the development process we took. Interactive projects typically don’t reveal how they work so we really hope other creators will get a kick out of it!
What else are you working on, anything coming up that you’re excited about?
Well we just left Amsterdam after three and a half rainy but glorious years and our excited for our move to the US. We relocated to Boston/Cambridge and are enjoying being surrounded by some of the nerdiest people on the planet! We are currently playing with some completely new experimental techniques for interactive character animation which we can’t wait to use on our next project!
Links:
Image Gallery
Boards Magazine
Theo Watson
Emily Gobeille
Open Frameworks
Credits:
Nexus Productions
Digital Directors: Theo Watson and Emily Gobeille
Design by: Emily Gobeille
Technical Development: Theo Watson
Executive Producer: Chris O’Reilly and Cedric Gairard
Production Manager: Alice Wigley
MOST Original Soundtracks and Audio Post Production
Sound Supervisor: Ivo Witteveen
Music and Sound Design: Ivo Witteveen, Otto van den Toorn, Diederik Idenburg
Audio Post Production: Stephan Bakker