# WEKA Color Correction During Post Production Media studios need color correction that keeps pace with compressed film and TV timelines, as finishing now spans 720p through commercial 8k and a different color space for every platform. **Type:** Solution Brief ## Introduction Today’s media studios depend on an extremely high-quality visual experience across many different platforms. They operate both on-premises as well in cloud environments to meet the needs of artists around the world and to allow flexibility in where they hire artists and scale their needs appropriately. One of WEKA’s customers is one of the largest studios in the world, with decades of experience, and is a leader in adapting to the desires of their customers in wanting richer and more immersive visuals in the stories they tell. They produce theatre, television, and streaming content, with a wide array of subsidiary properties. ## The Challenge The studio has very compressed timelines from ideation of the movie until the movie or TV show is released. The longer it takes to complete each step in the process puts more pressure on later stages and can increase costs significantly. Additional pressure has been put on the studio by technological changes such as upscaling rendering resolutions to meet the needs of different platforms: 720p, 1080p, 4k and even commercial 8k projection. One of the newer challenges is rendering in stereoscopic form for VR glasses. In addition to the rendering and transcoding processes, the finishing steps for the content include color correction and color space application for each distribution platform or different global region. This creates a multi-stage pipeline before media can be finished and sent out. Creative team priorities have made it hard to remove these pressures up till now. All of this needs to happen with real-time playout of uncompressed data from storage into the tools used by these finishing applications. According to the lead staff engineer for storage used during the process, “I need enough burst performance to handle unexpected loads, like the render farm hitting the storage with a hundred jobs at the same time. Having the machine falling over and the color correctors calling me asking why we’re stuttering on playback? That’s the call no storage engineer ever wants to get during post production.” The environment being supported includes Nucoda color correction systems, Clipster DDR and other transcoding tools as part of a Non-Linear Editing (NLE) workflow, a Nuke compositing system run on a render farm, and other tools sending data into the post workflow as well. Some of this other data includes management of Digital Cinema Packages (DCP), Academy Color Encoding System files (ACES), and Edit Decision Lists (EDL) to maintain tracking and consistency of output across the entire post workflow. The workflow also has to manage different color formats, depths, and spaces, different distribution formats and resolutions as well. All of the video until final packaging is handled uncompressed real time over the network. With an uncompressed 4k video stream representing over 1GB/s throughput, and many of these streams running simultaneously for each application on each workstation in the environment, this is no simple task. ## The Requirements To meet the expanding needs of the studio post-production workflow, they needed an environment that would centralize all post-process data into a single repository that can be shared by the color correction team, post admin team, rendering, transcoding for different distribution formats, all the video processing pipeline applications, and ancillary workflows. The technical requirements started at a baseline but needed to be future-proofed to scale as both resolutions and the number of color correction and rendering workstations increased. Initial requirements were: - High-performance concurrent access of 24Gbit/sec (3GB/sec) per station across a mix of different color correction stations and NLE finishing machines. - Multi-protocol connectivity to workstations and applications running on different operating systems. All protocols need to simultaneously access all data in order to simplify the data pipeline and prevent having to copy data in order to work on it. - Ability to perform as industry moves from to higher resolutions such as 4k to 8k, a quadrupling of the data needs for each video stream. - Performance delivered over SMB using SMB direct (RDMA), NFS for Mac workstations and the render farm, and POSIX connectivity for Autodesk RV and a custom pipeline app for compression to jpeg2000. SMB performance is key as many of the software tools used are based on Microsoft Windows OS. - Enough performance capacity to handle unexpected bursts from critical priority workloads. Must have 100% reserved performance capacity to meet these bursts without impacting existing work in progress. ## The Solution ![WEKA Data Platform workflow: video, color profiles, and composite images feed AVID, Nucoda, DaVinci Resolve, Clipster DDR, and Colorfront Transkoder over SMB, plus a Nuke render farm over NFS](https://cdn.sanity.io/images/ult5g8gw/production/d47973e7553b38d53488b42cc837fe0d1a9ba6d3-896x370.jpg) The WEKA Data Platform was crucial to designing the next generation of data movement in the studio. The studio deployed an on-prem WEKA system next to their color correction and rendering systems consisting of 10 Supermicro servers each with 5 drives providing over half a petabyte using 15TB drives and Cx-6 100Gbe cards, connected via ethernet through Cisco 100Gbe switching. The workflow application workstations also have either 40 or 100Gb cards to enable the throughput needed for the various processes. WEKA has delivered over twice the throughput needed to each station (actual results: 8GB/s per color correction station), ensured that additional stations can be added to increase the number of artists working at the same time, and has the flexibility to expand easily as data requirements increase. > “Color correctors doing work can’t have problems reading frames back, and they need to do it at 1GB/s on each machine,” the lead staff engineer said. “Luckily, we bought storage from WEKA. We can play back 2 8k streams at the same time on Windows, out of the box, using a single NIC with no special software or tuning on the windows machine. Literally, just map a drive on SMB, verify you’re using SMB Direct, open the app, click play, and 8GB/s performance just happens.” ## The Benefits/Outcomes By consolidating on the WEKA data platform, the studio realized several technical and business benefits. On the technical side, the artists and editors are seeing benefit using of the RDMA in SMB. It minimizes CPU load on the color correction machines making sure all of the compute is focused on the color correction applications and minimizing any stutters when working with uncompressed video. By the lead engineer’s estimation, they are also “…seeing a 2-3x reduction in cycle times for transcoding, format conversion and finish processing compared to previous NAS storage that we were using.” On the business side, by consolidating all data on WEKA, it flattened the infrastructure and removed the need for separate primary storage caches for color correction and prioritizing the use of these caches for critical needs depending on the project. This consolidation helped reduce administrative overhead and simplified workflows. Over the last several months the WEKA system has proven to be so fast that “When performance issues do pop up, all questions about whether slowness is associated with storage have gone away” Another benefit has been future proofing of the infrastructure. With the rapid pace of innovation in the visual effects and media creation space, there’s always been a worry about if what is currently available in their datacenter can meet future demands. “I have no worries about running out of performance capacity, we’re well positioned for more formats and more simultaneous projects” the lead engineer said. “We’ve chosen a storage system with enough performance and scalability that I’m not worried that our storage platform for post (WEKA) can keep up with current and future workloads. It’s just not a variable I have to worry about anymore.” ## Next Steps Learn more about how WEKA delivers faster project times and reduced costs for media and entertainment leaders. [https://www.weka.io/data-platform/industries/media-and-entertainment](https://www.weka.io/data-platform/industries/media-and-entertainment) [Download PDF](/api/resource-pdf?slug=color-correction-during-post-production)