SXSW 2013: What the Leap means to Motion Graphics


I had read about the Leap Motion Controller, but hadn’t seen it in action until SXSW 2013. The super precise motion-sensing device can track up to ten fingers within a three-feet area. Setting aside the obvious opportunities that the Leap opens up for those of us working in the interactive/experiential field, I’d like to focus on how the Leap might enter the workflow of a motion graphics artist.

There are three main ways the Leap’s ability to interpret 3D gestural data will allow for visuals to become ubiquitous that had previously been work intensive or cost prohibitive – as a 3D motion capture device, paired with a 3D sculpting app, and paired with a 3D drawing app. But perhaps the most exciting developments are happening when you modify the Leap to create home brew touch-screens.

Motion Capture for Keyframes

After Effects’s motion sketch is an oldie but goodie that I often go back to when keyframe velocities just aren’t hitting the timing quite right. More recently, the KinectToPin app has allowed for grabbing major joints (knees, elbows, etc.) as motion capture xyz positional data.

The Leap can grab the xyz of ten fingers at once, and lets you work from your desk (as opposed to the Kinect which requires space for the entire body). This could allow for quickly grabbing motion paths using your hands – in effect, creating keyframes in a way that feels more like acting or playing an instrument than point and click.

Perhaps we will use two fingers to experiment with creating a funny character walk, or animate a darting hummingbird with a back and forth of the wrist, or have the camera shake a specific way. This can all be done from scratch, but having the option to quickly motion sketch an idea and then finesse it will be an interesting alternative that may be speedier or create more interesting variations than a blank canvas. Here’s hoping for an z-position option in a future version of AE motion sketch!


Leap Motion Processing Library from onformative

3D Sculpting

One of the demos at SXSW 2013 was clay sculpting, reminiscent of Autodesk’s 123D Sculpt iPad app. Video experiments have already popped up of Unity, Blender, and AutoCAD integration, and I imagine eventually we will be able to get these inputs into Mud Box, ZBrush, etc.

My initial concern with digital sculpting is that instead of getting feedback through your hands, you’re getting it through your eyes. Even when “sculpting” on a Wacom or iPad, you still have the pressure of the stylus to a surface. The Leap allows for multi-touch zooming and rotating while pinching and pulling, but equally important will be how to dynamically and intuitively switch between tool types and controlling the strength of your sculpting tools.

I have yet to see any 3D models built with the Leap that showcase how it can beat a tablet/mouse + keyboard in terms of detail, but I’m not going to write it off yet. Perhaps the Leap’s most helpful role in this area is not necessarily replacing previous input devices, but supplementing them so that you can reserve your stylus/mouse for your sculpting tool, while your other hand gestures through menus or modifies camera views.


The Leap Motion experience at SXSW 2013 from The Verge


Leap Motion hands-on from The Verge

3D Drawing

In the Leap Motion hands-on demo above, and the 35 second mark of the Leap Motion Processing Library at the top of the article, you see a kind of 3D drawing/sculpting hybrid. By tracking the path of a user’s fingers, a model is created that’s a drawing in 3D space, reminiscent of Kinect and iPhone-enabled experiments such as Kinect Graffiti Tool, Graffiti Analysis 2.0, Movosity, and AirPaint. If these quick sketches could be easily exported as an .obj (or maybe even .fbx with animation) they’d provide an intuitive way to create organic looking 3D shapes quickly and easily.

I think this sort of 3D sketching has the most potential for creating new visual styles. For promising precedents, see Amit Pitaru‘s Rhonda and Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh’s EverybodyLovesSketch.


Amit Pitaru, Rhonda, (2003-1010)

But what about the day to day?

The three examples above indicate visual trends that could arise from the introduction of the Leap, much the way tools like Plexus, C4D’s Mograph, or Trapcode Particular allowed for visuals that were previously labor-intensive to become ubiquitous. The trick with any technique-based visual is that you’re using them on a job-by-job basis. Not all jobs require AE’s Puppet Tool, but when you need it, you’re really glad it exists. My last thoughts on the Leap would impact everyday, regardless of the brief you’ve been handed.

Jared Deckard’s experiments mounting the Leap upside-down to create a touch screen and creating an impromptu $70 Cintiq were the first tests that screamed “killer app”. If Adobe were to get on board with this type of Leap integration, I imagine it’d be a no-brainer for most artists to pick up a Leap to supplement their mouse and keyboard. Even for non-professional artists, combining the Leap with a desktop version of FiftyThree’s Paper could build a huge user base.

Deckard has a Javascript/HTML5 demo up of his open source graphics editor at inkmotion.org. I haven’t been able to test it, as I don’t have a Leap, but I’d love to hear about the experience from other developers (especially if you’re a tablet user).


Leap Motion Draw / Brush Demo by Jared Deckard

The Leap is set to ship on May 13, and has already announced that it will be bundled with ASUS computers. Their approach of opening up their SDK to developers and creating their own app store was a smart move in not going the way of the CueCat.

Gesture-based input technology, be it the Kinect, Leap, or some even newer-fangled thing, is going to become as common as the tiny video cameras that are now housed in every phone, laptop, and desktop monitor. Whether or not it’s a better (or even appropriate) substitute for the tablet or mouse in a professional creative setting will depend on effective software integration and rigorous user beta testing. I see it as a helpful addition to our toolset, rather than a wholesale replacement.

What do you guys think? I’m mainly a Photoshop/After Effects/C4D user with a keyboard, Cintiq, and 3-button mouse at the ready. I’d love to hear more our readers (particularly CG modellers) how they see gestural inputs entering their workspace.

12 Comments

Latham

When I first read about the leap last year my first thought is how could I use it to create some cool mograph work. And now look! Awesome post.

Ben Kidd

Definitely excited to have a play with this, but will reserve judgement until I have actually interacted with it. Of course it relies on software manufactures or 3rd parties to build it into apps such as AE and Zbrush, Maya etc. I can certainly see that it could make certain tasks more intuitive, such as file browsing or sculpting, as supposed to click specific tasks such as roto or vector illustration (I duno maybe they’d be fine too). All I know it sometimes I jump off the wacom and onto the mouse for when I need that “click”. Humans require feedback from devices, especially when taking on a task that requires accuracy and concentration. Leap could feel like a one way communication. I think the reason we use wacoms is because we like the feel of pens/pencil, leap doesn’t compare in that sense. I can see Leap becoming a desk addition rather than a mouse/pen replacement. Perhaps leap will become part of the keyboard or wacom tablet and thus disappear until user requires. Good luck Leap. Ben Kidd, CG Artist – Hamiltonkidd.co.uk

Julien

Powerpoint prez will take a new dimension… :)

Jokes aside, I can’t agree more with you Ben.
I still use my mouse for precision work. And the Asus partnership is predicting the merge of the device with future laptops.

I hope Leap will allow rigs or nulls manipulation directly in Maya, AE or Softimage. The animation process will become quicker and more intuitive.

I’ve pre-ordered a unit when they started to communicate about the technology. Can’t wait to receive it !

Maxwell

Really intriguing. I’ve always felt like clicking around with my wacom pen and mouse seems antiquated and clumsy at best.
Coming from a stop-motion background where you really do just reach out and grab things to move them, I’ve always dreamed of having that kind of direct interaction in the digital world. Will be interesting to see how it develops.

Felipe

well, ever had your director showing how he wants it animated using their hands? just ask them to do it over your leap motion.

sounds funny but we’re really close to it!

RICHARD

I was hoping to use the leap motion to replace mouse in after effects, I hope this is possible. Imagine a stage, that stage sets on top of the leap motion, but that stage is also your scene where by hand you can rotate,move,pan…etc any object/camera.

Vladimir Tomin

Is it possible to turn it into 3D scanner?

Ben Kidd

I wouldn’t be surprised, it looks like its drawing a point cloud. It’ll get hacked to hell and back so who knows what else we could see coming from the Leap platform. Long live the maker revolution!

Michelle Higa Fox

oops, you guys are beating me too my next article – 3d scanning and photogrammetry in the workspace, haha.

corrieparks

For conventional 2D animators (like me) the problem is going to be that I can’t imagine trying to draw in the air. I need the resistance feedback of the “pencil on paper”, which I now get with my cintique.

This looks like an awesome tool for anything that needs to be manipulated in 3D space, – though again, we are so dependent on touch feedback. But I imagine it will be similar to when pre-cintique graphic artists learned to look at a monitor while drawing on a wacom tablet. Our brains can get used to waving our hands in the air and watching stuff move on a screen. But when they invent some way to give us that physical feedback, that will revolutionize the way we do 3D animation.

Will

I love the idea of this being used in motion graphics for strictly a timing thing. As well all know timing can be rough and having natural speed and dynamic paths could really add to my (along with others) work flows.

Kev Chan

someone create a spherical holographic projector, then I can become Tony Stark

Comments are closed.