Henson Digital Puppetry Studio.

Redefining animation and interaction through real-time performances

What is the Henson Digital Puppetry Studio?

Integrated with motion capture and Unreal Engine, Henson puppeteers use the award-winning Henson Digital Puppetry Studio to create complete CG character performances in real time. This includes the puppeteering of full facial animation for characters and creatures. With the Henson Digital Puppetry Studio, we can “shoot” CG scenes as if they were live action and bring the magic of Henson puppetry into the CG world. 

‘Earth to Ned’ Uses Real-Time Digital Puppetry for Final Pixel TV

Components to Henson Digital Puppetry Studio

Henson Performance Control System.

The core of the Henson Digital Puppetry Studio is the Henson Performance Control System. An evolution of the same system that runs animatronic puppets, the Henson Performance Control System allows puppeteers to perform CG characters with the same creativity, spontaneity, and fluidity that they bring to the performance of live-action puppets.

The system empowers the puppeteer to design the animation style of a character. Custom hand controls and foot pedals give the puppeteer 32 programmable dual-direction inputs. The puppeteer gangs together individual movements into expressions then assigns those expressions to the controls. The puppeteer can activate those expressions individually or combine them to create additional expressions and nuance.

Motion Capture

The Henson Performance Control System integrates into any motion capture stage. To capture a full body performance, the puppeteer works in tandem with a body performer (or mocap puppet). An optical tracking system or an inertial sensor suit captures the body animation. The data from motion capture streams together with the facial animation from The Henson Performance Control System. They combine in real time to create a complete character performance. 

Virtual Cameras

Henson’s virtual camera controllers are designed with the functionality and feel of traditional TV pedestal cameras. Each camera can zoom, pan, tilt, crane, and dolly. Operators use the customizable buttons to preset shots. With three cameras operating during a mocap session, we can “shoot” our CG animation as if it were three-camera live action sitcom. 

Henson Take Recorder.

The Henson Take Recorder offers unparalleled flexibility in capturing and refining takes. It captures every input from the virtual cameras and puppetry performance control system. By isolating individual tracks (while preserving others), the DP can refine a camera position, the puppeteer can fix a moment of lip sync or an eyeline, the director can adjust a bit of movement. 

All Henson Take Recorder Data can then be imported back into Unreal and/or Maya for a post-production workflow.

Props Station.

The Henson props station is a slider box that can trigger prop actions and constraints. This allows us to trigger prop action or prop handoffs on the motion capture stage and reduces the need for keyframe animation for prop action and manipulation. 

Highlights

  • Word Party

  • Splash and Bubbles

  • Earth to Ned

    Earth to Ned

  • Sid the Science Kid

  • Hershey's Chocolate World's 4D Movie

    Hershey's Chocolate World's 4D Chocolate Movie

  • Jim Henson's Monster Dance

    Jim Henson's Monster Dance

Go behind-the-scenes and learn how we combined live action with full realtime CG animation using the Henson Digital Puppetry Studio!

Benefits

Spontaneity

There is magic that happens when creative people can improvise and experiment together. A shoot with the Henson Digital Puppetry Studio runs like a live action set. With the director, cinematographer, and performers all in one place, and with everything streamed in real time, the creatives can be nimble. They can explore different approaches to the script. They can iterate quickly. They can discover things in the moment to make the story better.

Context

Realtime rendering of the environment, lighting, performance, and cameras on the stage means that the director, cinematographer, and production team have full context for creative choices on the set. They can see, moment-to-moment, the full performance, camera view, and environment and make changes as necessary. When those takes go to editorial, the editor, director, and producers have full context for the cut. They can review, make creative choices about, and approve an edit that is very close to the final version of the story. This avoids expensive changes in the post-production process.

Perfect for

Realtime Animation

Visual Effects

Previz & Animatics

Live Interactive Experiences

Video Game Cinematics

Efficiency

With the Henson Digital Puppetry Studio, we can “shoot” 10-12 minutes of script in a day. The takes (with their underlying animation and camera data) provide a big leap forward for animators, sometimes eliminating the need for keyframe animation entirely.

Character Development / Reference for Animation

Social Media Campaigns

FAQ

Please reach out if the following answers don’t fully address your questions.