Smart OTT Monitoring & Remote Control Technologies Reduce DevOps Time
July 16, 2021
The legacy VOD workflow comprised a mixture of 3rd-party and open-source ingest, encoding, packaging, and QC software applications accumulated over the last decade and a half. The system was designed to max out at a 25k VOD title library. That might have been enough to support a catalog 15 years ago; however, not in today’s world of ever-increasing demand for VOD content.
As an on-prem system, there was no possible way to increase capacity on-the-fly, so the legacy system was easily congested and very slow to process titles, resulting in movies and episodic video content missing their scheduled start times (affecting contractual SLAs). Furthermore, the dashboarding and monitoring were not optimized to find bottlenecks and chokepoints quickly. The Operations team was not able to effectively track assets thru the workflow
With the content team looking to expand the existing VOD library to 100k titles within the next 12 months, it was very clear that a full re-architecture was needed to streamline and fully optimize VOD content processing. Our client was looking for significant reductions in per-title processing time and cost while incorporating intelligent dashboarding and automated scaling to handle on-the-fly capacity needs.
STAND 8 OTT engineers migrated the legacy on-prem VOD systems and workflow to AWS’ cloud infrastructure to leverage serverless, lightweight and inexpensive lambda and step functions and parallelization of processing workflows and on-demand scalability.
Highlights of the new AWS solution:
A multitude of error-correction workflow and automated workflow retry mechanisms were implemented, which decreased human intervention and time spent on troubleshooting workflow issues
Serverless architecture and on-demand scalability allowing for massive parallelization of metadata validation, transcoding, QC, and packaging processing workflows
Incorporated AWS’ FSx for Lustre file system which offers various price points depending on service level agreements (lower cost for VOD titles with a license window start date in 7+days in the future AND higher rates for top-tier premium content (i.e. C3 content) that needs to get on-air ASAP)
Smart monitoring and exception-based reporting to address workflow issues in a timely manner
To avoid processing congestion and bottlenecks, VOD titles were prioritized by license window start date guaranteeing that those license windows that have already started are processed first, followed by those license windows that start tomorrow are next, and the windows that start the following day are processed next, etc.
The migration to an AWS cloud-based architecture allowed our client full control and complete ownership of the entire AV pipeline. The new VOD workflow brings significant cost savings by reducing the need to maintain and upgrade 3rd-party server and storage infrastructure and expensive license subscriptions. These cost and time savings allowed for near-limitless expansion of the VOD library while continuing to meet SLAs and ensuring that the existing catalog was optimized for our client’s new OTT product offering.
Lastly, by developing the solution using Agile software development methodologies, the team quickly iterated and proved out new and creative innovation ideas; previously impossible working with 3rd party vendors and their inability to improve their product in a timely fashion.