After monitoring overnight, we are resolving this incident.
Cause
The run time for one of our scheduled job’s became longer than it was scheduled for, which caused a spike in one of our databases load. This led to slow performance of that specific database and caused the performance degradation.
Prevention Steps
- We have improved that scheduled job’s performance
- We have implemented new alarms to be early notified in case of future similar occurrences and be able to solve the issue without any customer impact.