Whereby - Some users are experiencing longer than expected connection times – Incident details

The team has compiled a complete post-mortem of the incident, which is available for viewing at the linked document.

After monitoring overnight, we are resolving this incident.

Cause

The run time for one of our scheduled job’s became longer than it was scheduled for, which caused a spike in one of our databases load. This led to slow performance of that specific database and caused the performance degradation.

Prevention Steps

• We have improved that scheduled job’s performance
• We have implemented new alarms to be early notified in case of future similar occurrences and be able to solve the issue without any customer impact.

We implemented a fix and are currently monitoring the result. Based on early results connection times and recording/streaming events should be working as expected.

UPDATE: We will continue monitoring until Wednesday 14 June 2023, but all systems should be operational and working as expected.