Cloud rendering services are a form of remote working and of virtualising hardware-based processes that studios have traditionally handled in-house. Such services have been evolving for several years and the market has now grown competitive as off-site rendering companies continue to improve, specialise and extend their functionality.
GridMarkets is a rendering and simulation service that supports freelance VFX artists and studios that have complex, time-intensive projects to manage. The company develops a platform of submission plug-ins supporting the most commonly used software applications for rendering, animation and simulation. These include Maya, Houdini, Nuke, Cinema 4D, 3ds Max and Blender content creation packages, and several renderers including Arnold, V-Ray, Renderman, Redshift, Mantra and others.
GridMarkets designed and built their platform by collaborating with VFX teams and continuously gathering input from them to keep further development to the plugins relevant to their work. The company uses 32 vrtual CPUs @ 3ghz/core with 60 Gb RAM and Tesla K80 or P100 GPUs, and has access to thousands of machines. Digital Media World had the chance to talk to Co-founder of GridMarkets Mark Ross about what makes this service different to all the others, and how it works.
Plugins and Pipelines
“To set the company apart from its competitions, GridMarkets focusses on several factors, one of which is extreme scalability,” Mark said. “A facility may not have a need for massive render capacity and power on a regular basis but when they do, knowing that they can specify the necessary number of high powered machines to finish on schedule is critical. To be able to scale directly from the team’s hardware, Gridmarkets’ service has an API for integration into their render farm and pipeline management system.”
The platform is also simple to work with, both to set up jobs and to pay for them. The submission plugin allows you to submit from your creation software’s UI, choose a preferred renderer, number of machines and GPUs.
The pipeline and workflow are different for each type of software. For Houdini, for example, a special pipeline has been developed to manage cloud simulations that includes uploading the dependency tree and all simulation inputs, plus caching so that the sim output can stay in the cloud until processing is complete. This saves using local storage for repeated transfers of output data.
The Cinema 4D pipeline now works with Release 23. Some of the pipelines include a Nuke node to process frames through a Nuke script after rendering, and a FFmpeg transcode node to compile frames into a video file on the GridMarkets servers.
All of the associated data transfers are via GridMarkets’ purpose-built file management tool called Envoy that uses standard HTTPS for transfers to and from Google Cloud Storage, where the user’s project files and submissions reside in one location in individual account buckets. Envoy is authenticated using Google Service Accounts. Envoy optimises large file transfers for quicker up and downloads, and the workflow allows you to see all submissions to be processed for a project.
Secured Global Environment
Mark said, “By taking advantage of GridMarkets’ global coverage and cloud-based tools, a production can make their project details and challenges visible to a dispersed team in a secured environment. GridMarkets assures users of the security of the service, one of the company’s Number One preoccupations, by following MPAA guidelines in the development and operation of its platform, which means that all of our partners – renderer and other software companies, cloud providers and so on – must comply with those regulations as well.
“GridMarkets executes tasks via a partner network of secured Oracle and Google data centres. The plaform’s security measures focus on access to the network, machine security, authorisation, content management and transfer and monitoring. An integration with Oracle Generation 2 public cloud infrastructure was made recently in order to increase the platform’s reliability and security.”
Oracle built the Gen 2 infrastructure specifically to run enterprise applications and databases. It has tools and utilities for constructing new cloud-native and mobile apps on a unified platform and networking fabric. In GridMarkets’ case, the goal is to establish secure rendering and simulation services for VFX and animation customers who generally prefer to devote more of their time budget to creative iteration than to technical infrastructure.
No Access
Purpose-built, secured virtual machines (VMs) manage the transfer of content. Each VM instance is only used for one job and its files and is then shut down to prevent access to the data by other users on later jobs. All jobs run with normal user permissions with no access to the machine’s administrative functions. Encapsulating the processing units in secure Docker containers within VMs also limits security risks.
No direct outside connection is allowed to any of the virtual machines to avoid exposing the compute nodes to the Internet. Data transfers and API calls within the GridMarkets infrastructure are all made through secure HTTPS connections – wireless communication is not used.
Managed Services and Pricing
“GridMarkets also operates managed services, which are mainly about saving time so that you can stay focussed on the creative demands of the project – and still meet your deadlines. The user delegates submission details to GridMarkets as if it was a member of the team,” said Mark. “GridMarkets then prepares the project files for simulation or rendering and keeps them together, helps manage the submissions of remote artists and optimises the scene and machine count so that the jobs process smoothly without stopping and starting.”
Pricing is handled entirely through pre-purchased credits, which are non-refundable but never expire. To upload and complete a job to the end, you need to have purchased enough credits in advance. A cost calculator is available online to help budget your project. Costs are calculated by machine hours, making it straightforward for on-demand use, but you can also cap your costs in different ways – limiting usage to a finite number of machines over time, for instance, or keeping render speeds down to a chosen level.
In early May 2021, GridMarkets discounted its services to Zync Render customers looking for new rendering options by offering a 20% discount for the first month of using GridMarkets. This offer follows the recent announcement that Zync is ceasing operations in June 2021 and supplies that company’s user community with a comparable solution in terms of features, security and reliability. www.gridmarkets.com
