Andy DehnartThe Masked Singer season five is the second season to be filmed since the world shut down one year ago, and it will be the second to be filmed without a live studio audience. Except just like with season four, there will be a fake studio audience, which is why the audience is not wearing masks, or social distancing in any way.
When Niecy Nash arrives on the stage—she’s replacing Nick Cannon as host for the first half of the season—in the premiere, the stage is surrounded by people clapping and cheering.
This is not an audience of actors; it’s a clever combination of awe-inspiring special effects and old-school editing. And now we have more information from the people who created those effects about just how it’s done.
First, during a virtual Television Critics Association press conference last week, I asked The Masked Singer’s producers and judges about why the fake audience exists in the first place, and if it’d continue in season five.
Executive producer and showrunner James Breen told me that “this season is going to be the same as season four, in terms of there’s a virtually created audience and there’s audience cutaways from previous seasons.” He said that’s “the best we can do, unfortunately, right now.”
He also said, “I think big, communal events are what everyone’s missing right now, and we wanted to keep that atmosphere and that fun for everyone at home. I think it would be a very different show without any audience.”
Nicole Scherzinger said, “I love it,” and Robin Thicke agreed. “We want to give people something that takes them away from reality,” he said. “We want to give them something to celebrate, great music and great times, and remind us that those times are coming soon again. It’s all about positivity.”
Robin Thicke also said that The Masked Singer has created a better phony audience than sports have: “When I watch basketball games, I miss that intensity. You miss that crowd. You miss the crowd’s reaction. You miss seeing how they would jump up and down to something, and that little bit of fantasy is necessary for us to fully dive into the experience.”
Jenny McCarthy-Wahlberg—vaccinate your kids, and get a C0VID-19 vaccine as soon as you can—said “cardboard cutouts just don’t cut it.”
Fair point, but other shows have filmed without an audience yet managed to capture the feel of a studio audience without resorting to fictional people. ABC’s Dancing with the Stars just used audience sound effects, while Food Network’s Tournament of Champions, which just returned, brought in a tiny but loud audience that all wears masks and sits far apart from each other.
Those approaches seem responsible to me, whereas I called the use of The Masked Singer’s fake audience last fall “a dangerous, unnecessary lie,” because the show never acknowledged—until the very end of the rapid-fire credits—that the audience was fake.
Tuning in to one of the most-popular broadcast TV shows and seeing a real audience suggested that, in the middle of a pandemᎥc, it was perfectly fine to gather unmasked, indoors, with strangers. And we all know that there are plenty of people who do think that’s true. Those yahoos refuse to do simple, kind things like wearing masks around other people, and that’s part of the reason why more than half a million people have died in the past year in the United States alone.
Breen doesn’t agree with me, at least not about his fake audience deceiving people. He said:
“I think people are pretty sophisticated, and I think everyone knows what’s going on in the world right now. So, I don’t worry that people think it’s real.”
I’m not sure whether to be in awe of his optimism, or send him links to tweets written by people who think it’s real, or just start crying.
Here’s the thing: What The Masked Singer has created looks very real and is quite believable. As Nicole Scherzinger said during the TCA press conference, producers “make it feel like a real audience.”
So they’re doing that again. But how do they do it?
How The Masked Singer’s set and audience special effects work—in real time!
Executive producer Rosie Seitchik said during the TCA press conference that, for season four and five, “we were really keen on holding onto that selling point and everyone’s ability to engage on that level and see everyone on the floor, engaging with the panel’s guesses and guessing who is under the mask.”
But as I wrote about last fall, it’s worth remembering that, from the very beginning, The Masked Singer has been faking its audience reaction shots.
Whether it’s during the actual unmasking or during a performance, you’re seeing footage that was probably filmed at a different time—perhaps while the audience was being coached to react, perhaps from another episode entirely. While the voting is real, even the shots of the real-life studio audience members voting on their phones are staged, as past audience members have revealed.
So how do they create the virtual audience?
The first part is easy. As showrunner James Breen told me, they’re using “audience cutaways from previous seasons.” What he didn’t say is how the virtual audience comes together—and it turns out that actually happens in real time, as they’re filming.
I learned about this thanks to an episode of the podcast Underunderstood, which interviewed two people responsible for the fake images. Co-host Billy Disney says that, in The Masked Singer’s credits, he noticed someone credited as “Pixotope/Unreal Programmer and Operator.”
They interviewed that person, Nickon Mirsepassi, who says on his website that he was an “augmented reality technical artist” for The Masked Singer season four. Nickon worked on the virtual sets, from the episode-one dinosaur to props on the stage. They’re virtually created with Unreal Engine, software that was originally developed for video games.
“You have a broadcast camera that’s recording everything that you normally would during a TV show, and then superimposed onto that our computer graphic elements that essentially exist within that set,” Nickon said on the podcast. “So the camera is being tracked in 3-D space, and based on the location of that camera, you can tell where the 3-D graphic elements should be, and how they look like inside of the camera. And this all happens in real time. You don’t have to composite this afterwards.”
That’s incredible: In other words, The Masked Singer isn’t adding special effects in post-production, but in real time.
Even more incredibly, he connected the actual set’s lighting with the computer graphics, so the lighting operator controls both lights on the stage and lights on the virtual stage. “If you give the lighting programmer control, he knows what lights he’s turning on in real life. So you can use the virtual equivalent lights to light the props and the set for the virtual props,” Nickon said, which means “it’s almost as if it’s sitting there in real life.”
The podcast episode goes into technical detail about all of this, which focuses on the stage and props. The fake audience, however, was added by a different company.
Silver Spoon, which worked with Fox Sports to create virtual crowds for Major League Baseball, also created the virtual audience for The Masked Singer. Underunderstood interviewed Silver Spoon executive producer Laura Herzing, who said the baseball crowds came together “in a matter of just a few weeks, we developed an entire virtual crowd platform to allow us to place, you know, 40,000 people in a venue.”
They used that system for The Masked Singer. What’s absolutely stunning is that this, too, happens in real time, just like the fake stage decorations. Herzing told Underunderstood:
“For all of our virtual audience applications, we have an actual operator there on site, who’s watching the game or the show and triggering what the audience should be doing in real time as well. So, we have built-in animations for clapping, cheering, booing—any kind of reaction that a real audience would have.”
The only audience members who aren’t added in real time are the people we can sometimes see behind the judges, and that was filmed separately and added in post-production.
So The Masked Singer’s audience is a combination of staged shots of real people, and computer-generated people inserted in real time. As I mentioned, that’s continuing for season five, but what about after restrictions lift and it is safe to be back together?
Herzing told Underunderstood that she thinks the virtual audiences will still be used—though it’s worth remembering that her company has a financial interest in seeing them continue:
“Once people do come back to these games and these studio shows, I think that integrating augmented reality with practical set builds is just going to be kind of the way things are done. That’s the way that broadcasting is heading, it’s just it’s all going to be kind of mixed reality.”
So we may not be able to tell what’s real and what’s fake? I suppose that’s not unlike a lot of reality TV over the past 20 years.