Advertisement

Wonder Dynamics puts a full-service CG character studio in a web platform

The tools of modern cinema have become increasingly accessible to independent and even amateur filmmakers, but realistic CG characters (like them or not) have remained the province of big-budget projects. Wonder Dynamics aims to change that with a platform that lets creators literally drag and drop a CG character into any scene as if it was professionally captured and edited.

Yes, it sounds a bit like overpromising. Your skepticism is warranted, but as a skeptic myself I have to say I was extremely impressed with what the startup showed of Wonder Studio, the company's web-based editor. This isn't a toy like an AR filter — it's a full-scale tool, and one that co-founders Nikola Todorovic and Tye Sheridan have longed for themselves. And most importantly, it's meant to make artists' jobs easier, not replace them outright.

"The goal all along was to make a tool for artists, to empower them. Someone who has big dreams doesn't always have the resources to manifest them," said Sheridan, whom many will have seen starring in Spielberg's film adaptation of Ready Player One — so his familiarity with the complexities of CG-assisted production and motion capture are very much firsthand.

Todorovic and Sheridan have known and worked with each other for years and frequently hit this wall: "Both Tye and I were writing films we couldn't afford to make," said Todorovic. Their company, which has operated mostly in stealth until now, raised a $2.5 million seed round in early 2021 and an additional $10 million A round later that year.

ADVERTISEMENT

The thing is, although software for creating 3D models, editing, compositing and coloring (among other steps in the filmmaking process) are much easier to buy and use these days, the process for actually putting a CG character in a scene is still very complicated.

Wonder Dynamics founders Nikola Todorovic, left, and Tye Sheridan, right. Image Credits: Wonder Dynamics (left) / Matt Winkelmeyer / Getty (right)

Say you want to include a robot companion for a scene in your sci-fi film. An artist making a model and textures and so on is only the very first step. Unless you want to hand-animate it (not recommended!), you'll need a motion capture studio or on-set gear, reflector balls, green screens and everything. From those, motion primitives need to be applied to the CG skeleton, and the character substituted for the actor. But then the 3D model needs to match the direction and color of the lighting, the cast and grain of the film, and more. Hopefully you hired people to capture and characterize those as well.

Unless they happen to be an expert in all of these individual pre- and post-production processes and have a hell of a lot of time on their hands, it's simply out of scope and budget for most filmmakers. At the end of the day you could be looking at as much as $20,000 per second for major VFX work, like adding a dragon or superhero, not to mention days' worth of technical labor. So indie films tend not to have prominent VFX at all, let alone fully animated characters.

Wonder Studio is a platform that makes this process as simple as selecting a filter or brush in Photoshop. It sounds too good to be true, but Sheridan and Todorovic have been working on it for three years now and the results show it. "We wanted to build something foundational — that's why it took so long," said Sheridan.

"We built something that automates this whole process, animates it live, frame by frame, there's no need for mocap. It automatically detects actors based on a single camera. It does camera motion, lighting, color, replaces the actor fully with CG," Todorovic explained.

But crucially, it doesn't just do this live in camera by pasting it on there à la TikTok, or spit out a questionably "final" product. All the actual pieces that VFX artists would normally create or interact with are still generated. And, he was careful to point out, none of it is trained on artists' existing work.

Final shot, mocap data, mask and 3D environment generated by Wonder Studio. Image Credits: Wonder Dynamics

"You get mocap, clean plate, masks, blender scene, it analyzes the noise and grain," he continued. That is: animation and motion data (including hands and face), the shot without the actor or replacement, outlines of characters and objects frame by frame, a 3D representation of the environment with terrain and other features. And it's all matched automatically to the qualities of the shot — or shots, as it can track actors across a full scene with multiple angles.

Here's a look at how the process works in action:

Lots more examples and details can be seen at the company's site.