We have seen all systems stabilize in line with the cloud provider reporting their systems have stabilized and are operating normally.
As part of this incident, we wanted to provide more detail about the impact of this incident on our services.
The primary impact this outage had on Zype's services was on our playout and VOD manifest delivery systems.
Playout: Stream playback experienced occasional buffering and playback interruption on multiple endpoint providers such as PlutoTV, The Roku Channel, and LocalNow. Other providers may have been impacted as well. In addition to playback interruptions, we saw intermittent timeouts, black screens, and audio silence errors reported on streams.
VOD Streaming: VOD manifests experienced elevated error rates and intermittent longer load times. Consumers would have seen VOD videos take longer to load upon first opening the player.
Posted Dec 07, 2021 - 20:37 EST
Update
We are seeing performance stability in line with reports from the cloud provider that stability is returning across services. We're going to continue monitoring as progress is made.
Posted Dec 07, 2021 - 18:31 EST
Update
We are continuing to monitor for any further issues.
Posted Dec 07, 2021 - 18:12 EST
Update
We're continuing to monitor the hosted cloud provider outage for updates. We will publish an update to our status page as soon as one is provided.
Posted Dec 07, 2021 - 17:54 EST
Update
We are continuing to monitor the hosted cloud provider outage and will continue providing updates as the root cause is addressed.
Posted Dec 07, 2021 - 16:25 EST
Update
We are beginning to see improved response times and decreased error rates on VOD manifests and playout. We are continuing to monitor the cloud provider incident.
Posted Dec 07, 2021 - 14:59 EST
Update
We are now seeing impact to VOD streaming performance, as our VOD manifest system is experiencing elevated error rates in relation to this incident. Consumers may experience increased VOD load times and buffering throughout a VOD stream. We are continuing to monitor the hosted cloud provider outage and will continue providing updates.
At this time, we understand the root cause of the issue has been identified and the solution is actively being worked on.
Posted Dec 07, 2021 - 13:29 EST
Monitoring
We're starting to see systems recovering in relation to the hosted cloud service issue. We're continuing to monitor our systems.
Posted Dec 07, 2021 - 12:26 EST
Identified
We've identified an issue within some of our hosted cloud services causing intermittent playout streaming issues. Our team is aware and actively working on remediating the issue.
Posted Dec 07, 2021 - 11:15 EST
This incident affected: Playout and Manifest Bridge.