Pictured: Fish, from the 'The Boxtrolls' movie, and a child jump together. The Boxtrolls is a stop-motion animated adventure from Laika, the makers of Coraline and ParaNorman. It's the story of a young orphaned boy raised by underground cave-dwelling trash collectors, who tries to save his friends from an evil exterminator. The Lodge was tasked with introducing the Boxtrolls to as many new audiences as possible. We knew once kids and parents met them, they'd love them. We arrived at the idea of a "magic mirror." It's a portal that lets us peer into the subterranean world of the Boxtrolls, and they can peer back into ours. The Boxtrolls Magic Mirror allows you to interact with characters from the film. One of four characters from the film waits in their box on a large screen sitting in an ornate antique frame atop a stone base. Approaching the mirror brings the Boxtroll to life. As you move, pose, jump, and gesture, your Boxtroll friend will try to copy you. If you wave, he'll wave back. You jump; he jumps. If you flex your muscles, he'll show you his with a growl. He'll even try to copy the expression on your face. The Magic Mirror launched at The Boxtrolls Family Day at Universal Studiosbefore making itself at home in the lobby of AMC's Universal CityWalk Theatre in Los Angeles. Pictured: The Magic Mirror strives to closely resemble the look of the film. Laika decidedly limits the use of computer graphics in their animation. There is something amazing about the depth and detail that comes from handcrafting every frame of their films. We knew if we were going to be rendering the world of the Boxtrolls with a computer in real time, it had to look convincing. The Boxtrolls Magic Mirror software was made with the Cinder creative coding framework. We used a prototype branch of the library that enabled us to develop a top-notch graphics-rendering engine. Taking notes from techniques used in AAA gaming titles, we wrote an optimized deferred shading pipeline leveraging new features in Cinder. That meant we could do more with lighting and effects than we would be able to with a traditional graphics pipeline. The end result is a real-time rendered image that closely matches the handmade look of the film. Pictured: The Microsoft Kinect was used to control the Boxtrolls. The obvious choice for interacting with the Boxtrolls was the Kinect. Near the start of the project, we worked with Microsoft on the Cinder implementation of the Kinect Common Bridge; a user-friendly library that operates the device. This relationship was instrumental in pushing the limits of the sensor. The Kinect v2 had just come out, but its software wasn't mature enough to implement on this project. Since then, the Lodge has released full-featured open source Cinder libraries enabling both generations of Kinect. Pictured: "Sampling" is a fast and versatile C++ data processing library developed by The Lodge. It was the foundation for stabilizing Kinect data and creating pose and gesture recognition engines. The Kinect's data stream was very noisy. Trying to track motion with the raw data resulted in a choppy and jolting experience. We had recently created a library called "Sampling" which enabled us to quickly implement fast filtering to stabilize the data and build a gesture recognition engine. We could boil body movements into simple actions like "clapping hands," "covering eyes," and "jumping", and recognize face poses as "say ah," "wide grin," and "pucker." Pictured: We worked with Laika to create interactive CG versions of their practical models. With stable real time data and gesture recognition, we could bring the characters to life. The workflow for creating an interactive model is typical in a 3D gaming studio, but not so much in an environment focused on practical animation. Laika created full CG versions of a few characters from scratch, complete with texture maps that appear to have been meticulously hand-painted. We worked with our motion team to make them interactive. Pictured: Facial expressions are accomplished by blending multiple head models into one. We operated the face differently than the body, so the heads were made separately and then reassembled in our software. We gave Laika a list of controllable facial features and they returned a series of models representing each. We turned these back into single meshes which blend the values of each feature to form an expression. Furthermore, the eyes work independently of the head. Blink rate, eyelid openness, gaze position, and tropia (which is how "crossed" or not eyes are) made the difference between a stale, robotic computer model and character that looks like he might be up to something. Pictured: Custom software was developed to pose and animate models. As the Boxtrolls came to life, we saw the potential to give them some autonomy. Our developers and motion designers worked together to develop a Cinder-based standalone piece of software for posing and animating the Boxtrolls. We used it to export animation files in a proprietary format that was easy for the Magic Mirror to understand. Each Boxtroll is parameterized to perform these moves with their own personality. You can even play peek-a-boo with them! A walkthrough test of the interactive features. It's the little details that make the difference. We tediously mined the vocal tracks of the entire film to select snippets of audio for each character. When you say "ah", so does he. A Foley session with a cardboard box and a pair of shoes yielded the recordings we needed to add sonic qualities to the minutiae of movement. If a Boxtroll sees you from a distance, his eyes will follow yours. You can use your head to control the virtual camera's position and get a better look at the cave. To round out the experience, background music winds up when you step into view. Pictured: A bug plots its path along the terrain. In the film, bugs are the Boxtrolls main food source. We heard on several occasions that "it would be cool to have some bugs crawling around." This "little" feature was quite an undertaking since the bugs have more complex skeletons than the Boxtrolls. Unlike the main character that just stays in one place, the bugs need to understand how to navigate the terrain and not bump into anything. Test video of a bug's walk cycle. We worked with our motion team to animate the bugs in a way that gave them their own personality while allowing us the flexibility to dynamically position them in the cave. Pictured: A little girl has the largest Boxtroll under her control. For the all the technology that went into this project, it succeeds because it hides it. The experience is kept at an emotional level. By giving kids and parents a one-on-one interaction with the Boxtrolls, the Magic Mirror accomplishes the difficult task of getting them to fall in love with the characters of an animated film before they even see it.