That’s something we never tried to produce internally. Up until now, Cirque du Soleil used similar effects but always and only with pre-recorded video with a single tracking point.
The idea was to use motion capture, mostly used in animation and video games, but on stage and in real time. To create this animated shadow, we worked with 4U2C, specialists in projection and visual effects, RodeoFX, experts in motion capture and virtual character creation and BlackTrax, a real-time tracking system.
The goal was to develop an avatar in 3D. The avatar becomes the alter ego of the artist, with his own personality, that could follow the artist everywhere on stage.
The 3 main steps
Step 1: Capture the shadow. Scan the artist in 3D with sensors to get a virtual point cloud and film the artist in the role of the shadow.
Step 2: Create the shadow.
Step 3: Link the shadow to the artist
Did some things go wrong? Absolutely.
We experienced bugs and technical problems around elements like visual occlusion, the speed of movement or movement of the artist, the fluidity of the movements, the interference between the real shadow and the virtual shadow and many more….
Did we learn from it? Absolutely.
It is vital to predict the scenery and the positioning of the cameras according to the acts and the technical needs of those. Furthermore, the costumes must be designed based on the technology that will be used. And finally, you have to choose the right kind of skeleton (rig) to get the best fluidity out of the avatar’s movements.
In the end, although the effect was only partially used in a show – the shadow follows the artist in real time but the interaction is recored. We now know the extent of the possibilities with the currently available technologies when it comes to live mocap.