OWNZONES’ DNS fight

At OWNZONES we pride ourselves on being very customer-focused and on scaling to the extremes so that we can meet the highest standards of the media industry. In this pursuit we sometimes hit particularly challenging problems that we have to solve for our customers. Months and months of dev-time are invested in stability, better detection for FrameDNA — because we got a new use-case, and many others.
Sharing the struggles that we experience with the community and being more open about the tech we are using is something that we have been trying to do for a while. We want to showcase how we solved or mitigated issues, the hurdles we’ve passed, and the fun we’ve had. We want to push the industry forward and by being more open about our practices we are taking a significant step to that end.
While stress-testing our systems, we started to notice that the runtime for specific tasks was longer than we expected. For example, we know that a full-feature HD movie at around 260GBs should take about 1 minute to transcode in ideal conditions. A UHD version should take about 16 minutes. When our testing department was doing exploratory testing and throughput testing, we saw that the transcode for the HD version was taking around 5 minutes, even in ideal conditions. We were seeing other parallel tasks take minutes instead of seconds. And we had no idea why — other than the scale, we were seeing nothing wrong.
After analysing the impact this had on our customers, we decided to create a cross-functional task force to investigate these issues. We made significant changes to our applications to get more visibility, we invested in new tools, and we aimed our attention at stability and reliability. We are really proud of how the teams worked together to fix this significant blocker for scale and how we evolved our application into something even better.
The whole story can be read on the AWS Container Blog, https://amzn.to/2NjUmc3, where we focus more on the technical aspect of DNS issues and solutions.
