Jessie Kim and Tim Lambert are co-founders of the virtual reality production company Prosper XR, and the creators of XR Alliance, a VR content competition intended to help overcome one of the main challenges that many potential AR/VR content creators face – the search for suitable stories and scripts.
Digital Media World talked to Jessie Kim about her and Tim's company, about their upcoming XR Alliance competition, and about working in the VR production industry right now. In the past year, she has produced about 18 VR and AR experiences, taking roles from director and director of photography to VFX supervisor. She also wrote the Daydream Impact 101 curriculum for Google and mentored NASA/JPL in Google's first VR Creator Lab. Prosper XR has created content for brands such as Disney, Live Nation, Hulu, Google, Buzzfeed and Netflix.
VR Native
When looking over the company’s projects, you'll notice that most of them seem to be natively VR-driven - not simply movies, games or rides re-engineered for virtual reality, but experiences and stories that would be difficult or impossible to conceive in any other way.
Jessie believes it's critical for producers to seek out and develop stories and projects specifically aligned to the VR medium from the start. It's often hard to break away from traditional ways of approaching new project ideas, but that difficulty could be a result of less-than-ideal first-time VR experiences. If she could, she would make sure that everyone’s first, second, third VR experience is comfortable, emotionally impactful and enlightening. “We never forget our first time,” she said. “My first experience using a VR headset made me want to quit my full-time job and throw myself into exploring immersive mediums.
“As a producer, the first question you need to ask yourself when evaluating a project is, 'Why does this have to be in VR?' Good virtual reality enables users to feel embodied presence, to be teleported into the world that you create for them. Examples include emotional stories that envelop and unfold around the user with strong character performances, guided tours of exclusive locations, interactive experiences that enable users to do things that they wouldn’t normally have the opportunity to do.”
Jessie Kim, Prosper XR.
Elusive Experiences
Producers lacking experience in immersive mediums like VR can benefit from finding experienced people to team up with. Technically proficient and creatively talented people are equally valuable to help you get started. Jessie said, “In this nascent industry, many people try to do everything on their own, and I think that is a big mistake. Technical people are not typically the best storytellers, and some of the most creative artists and storytellers try to cover all the technical processes themselves. Furthermore, a master of traditional mediums may be at a disadvantage compared to a new filmmaker because the language of creating, developing and reaching audiences through immersive content is so different.”
She doesn't find it too surprising that the answers to questions about what makes great VR and great user experiences remain elusive. “We are all pretty foreign to this medium. We grew up with flat screens on our walls and desks and in our hands. We’re used to shots incorporating close-ups, pans and zooms, and experiencing them seated with friends and family, but young people today who are growing up with immersive content are 'immersive natives' and gravitate to it naturally. As they become content creators, they will make a critical impact on the development of immersive media.”
XR Alliance – Seeking Pioneers
Before sharing some of her practical, working experience in VR production and post, Jessie talked about the XR Alliance and competition, which is in fact a search for immersive content and scripts. Contestants will have their scripts read by a panel of judges made up of experienced VR, AR and mixed reality producers, directors and developers. Winners will be invited to join the Alliance's XR Development Lab, where they will be matched with an experienced mentor and have a chance to continue developing their idea into a finished project.
Jessie said, “As founders of the XR Alliance and its inaugural competition, we were inspired by the isolation that content creators were feeling in all parts of the industry, from big and small companies, to freelancers and creatives. We wanted to form a group of pioneers from the many different sides of immersive media to inspire, connect and collaborate.
“Our partners, SpecScout.com, run a database and content management system built by Tim Lambert, the CTO of Prosper XR, that we are now using for the XR Alliance competition. With access to this infrastructure that has been working in the traditional entertainment space, we are now using it for immersive scripts."
A technologist at heart and a creative second, Jessie readily understands the technical side of immersive production. Nevertheless, she noticed that while some amazing technology demonstrations have been on view, no one really cares about such advances if they aren't being used in a context of impactful storytelling.
“For example, a company developing software for volumetric lighting asked people watching their demonstration to notice the how the shadows changed on a rock as they moved their heads,” she said. “But most people don’t care about tech for tech’s sake. They want to go on an adventure and follow a story that moves them emotionally, takes them on a journey and transports them to another world, whether that story is linear or interactive.
"I've noticed many talented creatives in the industry writing scripts and developing great projects, but who don’t know what to do with them to bring them to life. I wanted to create an outlet where those producers can be recognised and surrounded by the people who can help them the most."
VR Development and Production
When identifying some of the main production hurdles nearly all producers encounter, starting with development, she noted that writing a script can also pose challenges in communicating new, complex concepts to a team and crew that only those with experience consuming VR can understand. For instance, how does a script writer effectively communicate a branching narrative with interactive elements to everyone associated with the project?
There are also new mediums to consider integrating into the immersive experience. In Augmented Reality, the real world becomes part of the experience. Mixed Reality incorporates real world objects into the experience so that, for example, users can reach out to touch a furry animal and actually feel fur, and Artificial Intelligence allows characters to respond to the users’ actions. In the meantime, a common language that everyone can comprehend still hasn’t been developed to incorporate all of these possibilities.
Tim Lambert, Prosper XR.
Once a script is written, how should you reach out to find funding for a project? Raising funds for any type of production is very difficult today, and many VR projects don’t make it past the demo phase.
Jessie said, “In production, trying to film in custom-built immersive sets can get expensive. Crews must learn to hide lighting and rely more heavily on practical lighting. There is still no perfect VR camera - all have their benefits and challenges. With regards to distribution, scaling the experience so that enough people get many views is a further challenge. Beyond showing at film festivals, your user base or audience may not have immediate access to a VIVE headset and instead choose to watch it on their phones via YouTube or Vimeo, which may compromise the effect. Apps like Hulu VR help ensure that users watch natively in VR and have a richer experience.
Post Production Pipeline
“When people get involved in VR post production for the first time, they quickly realise that they need to seriously invest in upgrading their infrastructure to handle editing at resolutions of 6K or higher,” Jessie said. “For post production to go ahead as smoothly as possible, an efficient data management infrastructure with RAID backup and machines using the latest graphics cards is important.
“The first generation of tools now in use for post production, on both the hardware and software side, can be buggy, clunky and unrefined. Much of software is new - there are some great tools out there like Mistika for stitching, Mettle for applying 360 effects (now incorporated into Adobe After Effects) and Mocha VR for compositing – but so far no software is completely polished and bug-free. The best programs have a steep learning curve and may crash frequently without the proper infrastructure in place to run it effectively.
“When production has been done by an inexperienced crew, every shot potentially becomes a VFX shot. Stitch lines may need to be fixed, and rotoscoping, stabilisation, rig removal and compositing all increase post costs far beyond what most people anticipate when starting out.”
She also notices that although many traditional media film and television folks are now getting into VR, and may be pitching projects and securing funding, they are still not allocating realistic budgets that VR truly needs to churn out good content and have a lasting effect in the market.
Furthermore, some traditional media DPs do not fully understand the camera systems yet or the challenges of post production. 'Fix it in post' is an especially dangerous mindset on any 360 video VR production. “Otherwise, post costs can get out of hand,” remarked Jessie. “For example, on a recent shoot, the crew put a stereoscopic camera with a large nodal offset – that is, with the 24 cameras positioned quite far apart from each other in the camera array - into the tight space inside a vehicle. Due to parallax from the orientation of the cameras, the images proved nearly impossible to stitch together in post, and the production actually decided it would be far less expensive to reshoot with a different camera set-up."
Rig Rover
Mainly a VR/AR and MR production company, Prosper XR also focuses on innovation. Currently, for example, the company is developing a line of robotics and Rovers that will open up possibilities for new types of camera movement to directors in this field.
As early VR content creators, they regularly run into problems posed by a lack of tools for camera work they particularly want to undertake. Consequently, to some extent they have had to create the tools themselves to accomplish their projects. One such problem from the start was how to move a camera smoothly.
Tim Lambert with the Rover Rig.
“When we developed our first Rig Rover, we had the only Rover in existence for VR. Over the years since then, we have continuously developed and iterated our robotics, mainly driven through the needs of the content and of our collaborators in the industry,” Jessie said.
“We created the only VR Rover that allows you to simultaneously capture sound and video at the same time, so we do a lot of actor performance capture projects. Using custom-made horizon stabilising gimbals, we can prevent yaw movement when turning the camera, so the camera always stays in the same rotation plane regardless of the change in direction of the rover. Yaw movement can instantly create VR sickness.”
Users can also use the rig to raise and lower the camera remotely. Development priorities are a small footprint, extremely quiet functioning, anti-vibration and shock absorption. The rig is compatible with commonly used and new professional VR cameras. Recent sample shots from the Rover can be viewed here. prospervr.com
