Website, Video playback, Search, Recording / Uploading, Processing / Encoding, Integrations
September 27, 2024 4:01PM UTC
[Resolved] RCA: On the morning of September 23rd from 01:00 UTC to 06:00 UTC, the Panopto AP environment experienced a service degradation, including site slowness and increased error rates. We identified degraded database performance during a period of heavy load on the system as the root cause. We apologize for any inconvenience this issue may have caused. In order to prevent this in the future, our engineering team will be performing scheduled downtime maintenance in order to increase the database CPU and memory capacity. All affected customers will be contacted directly via email in accordance with Panopto's downtime notification policy.
September 26, 2024 7:51AM UTC
[Resolved] The AP Panopto site continues to be fully operational. The Panopto Cloud Operations team will continue to closely monitor the system.
We will post an RCA by Monday, September 30th.
September 24, 2024 11:59PM UTC
[Monitoring] All site metrics remain in a healthy state. We are continuing to closely monitor the site and investigate the root cause.
Our next update will be within 24 hours.
September 23, 2024 7:09PM UTC
[Monitoring] All site metrics remain in a healthy state. We are continuing to closely monitor the site and investigate the root cause.
Our next update will be within 24 hours.
September 23, 2024 12:59PM UTC
[Monitoring] All site metrics remain in a healthy state. We are continuing to monitor the site and investigate the root cause. We are sorry for any inconvenience this may have caused.
Our next update will be within 6 hours
September 23, 2024 7:00AM UTC
[Investigating] The site slowness and errors have improved significantly. We are continuing to monitor the site and investigate the root cause. We are sorry for any inconvenience this may have caused.
Our next update will be within 6 hours
September 23, 2024 6:04AM UTC
[Investigating] We are investigating site slowness and increased error rates on the Panopto AP Cloud.
We're sorry for this interruption, our cloud operations team is urgently investigating the issue and we'll update here as soon as we know more.
Our next update will be within 30 minutes
September 23, 2024 4:24AM UTC
[Investigating] We are continuing to investigate the site slowness and increased error rates on the Panopto AP Cloud.
We're sorry for this inconvenience. Our cloud operations team is urgently investigating the issue and we'll update here as soon as we know more.
Our next update will be within 30 minutes.
September 23, 2024 4:00AM UTC
[Investigating] We are continuing to investigate the site slowness and increased error rates on the Panopto AP Cloud.
We're sorry for this interruption, our cloud operations team is urgently investigating the issue and we'll update here as soon as we know more.
Our next update will be within 15 minutes.
September 23, 2024 3:09AM UTC
[Investigating] We are investigating site slowness and increased error rates on the Panopto AP Cloud.
We're sorry for this interruption, our cloud operations team is urgently investigating the issue and we'll update here as soon as we know more.
Our next update will be within 15 minutes.
