Dear users and partners,
We want to update you on the recent service disruption you may have experienced. We understand that any interruption in security communication can be frustrating, and we appreciate your patience as we resolve this issue.
Upon conducting a comprehensive investigation into the incident, our initial analysis revealed no explicit causes for the outage within our logs and metrics. This situation has posed a significant challenge in identifying the root cause.
The disruption happened because one of our servers failed, and our load-balancing system couldn’t handle it in one go.
18:08. After receiving the first Arm/Disarm delay reports, we started deploying another server cluster.
By 18:45, we partially restored systems and app operations, but the abnormal database load prevented us from fully deploying server clusters several times every 30 minutes.
At 21:15, the third deployment attempt fully restored proper communication for all Ajax apps and hubs.
To ensure the continued reliability of our services, we are actively exploring various hypotheses, including the possibility of memory misuse, Linux core fatal errors, or untraceable issues within Amazon Web Services (AWS). Our best cybersecurity engineers investigate these avenues thoroughly.
❗ To assure you that Ajax functionality is not limited to the cloud only, all non-cloud tools are available, disregarding a cloud. Functions such as direct connection to the monitoring station, direct SMS/call from the phone, and the usage of SpaceControl and KeyPad to arm and disarm the system continue to operate without the need for a cloud connection.
We are also working on a policy regarding emergency notifications, failures, and other similar aspects to keep you informed about the state of affairs. However, we hope that there will be no need to use this policy.
Recognizing the importance of a seamless user experience, we are committed to delivering on that promise. Rest assured that our teams are diligently working to identify potential areas of concern and implement necessary improvements. We sincerely appreciate your understanding during this time. Your trust in Ajax Systems is invaluable, and we are dedicated to providing you with the high-quality service you expect.