status.amazee.io
Identified - An issue has been identified with one of the servers of the cluster. To resolve this requires a full reboot, we will wait until after 22:00 CEST to minimize impact of the reboot.
Oct 17, 18:27 CEST
Investigating - We are currently investigating this issue.
Oct 17, 16:53 CEST

About This Site

Issues on amazee.io but nothing shown here?
Let us know on RocketChat or support@amazee.io, and we'll help you right away.

Documentation? Find everything at docs.amazee.io
What is amazee.io?

General Operational
API   Operational
Deployment Infrastructure   Operational
Nameservers   Operational
dev1.compact   Operational
Switzerland Degraded Performance
zh1.single   Operational
zh2.single   Operational
zh1.compact   Operational
zh2.cluster   Operational
zh2.compact   Operational
ch1.lagoon   ? Degraded Performance
USA Operational
us2.compact   Operational
us1.lagoon   Operational
South Africa Operational
sa1.compact   Operational
United Kingdom Operational
uk1.compact   Operational
uk2.compact   Operational
Germany Operational
de1.compact   Operational
Finland Operational
fi1.compact   Operational
Japan Operational
jp1.compact   Operational
on-premise servers   Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Oct 17, 2018
Completed - The scheduled maintenance has been completed.
Oct 17, 02:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Oct 16, 22:00 CEST
Scheduled - We are conducting regular maintenance on our systems.
Oct 16, 17:51 CEST
Oct 16, 2018
Completed - The scheduled maintenance has been completed.
Oct 16, 15:49 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Oct 16, 15:00 CEST
Scheduled - We will conduct scheduled maintenance on jp1-compact.
Oct 16, 14:18 CEST
Oct 15, 2018

No incidents reported.

Oct 14, 2018

No incidents reported.

Oct 13, 2018

No incidents reported.

Oct 12, 2018
Resolved - The situation could be resolved. There was a high volume of illegitimate traffic causing high load on the server. Our engineering team went ahead and blocked the traffic on the network level.
Oct 12, 16:34 CEST
Investigating - We are currently investigating this issue.
Oct 12, 16:08 CEST
Oct 11, 2018

No incidents reported.

Oct 10, 2018
Completed - The scheduled maintenance has been completed.
Oct 10, 02:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Oct 9, 22:00 CEST
Scheduled - We are conducting regular maintenance on our systems.
Oct 9, 18:20 CEST
Oct 8, 2018
Resolved - This incident has been resolved.
Oct 8, 14:25 CEST
Update - We're currently investigating availability that affects some of the sites hosted on ch1-lagoon.
Oct 8, 14:12 CEST
Investigating - We are currently investigating this issue.
Oct 8, 13:57 CEST
Oct 7, 2018

No incidents reported.

Oct 6, 2018
Resolved - Upgrade is complete, the cluster is showing stable for the last few hours.
Oct 6, 04:16 CEST
Monitoring - We have completed applying updates and doing the final reboots of all systems. We will monitor the cluster for a while longer before calling this emergency maintenance concluded.
Oct 6, 02:08 CEST
Update - All compute node updates have been finished - Based on monitoring all sites are back online again. We're conducting the post upgrade checks at the moment.
Oct 5, 23:20 CEST
Update - We will continue with the OpenShift upgrade of the compute nodes
Oct 5, 19:58 CEST
Update - Control Plane update is fully done now and the cluster is stable. We'll move on to upgrading the compute nodes after 8pm CEST in order to move the emergency maintenance further out of business hours.
Oct 5, 18:09 CEST
Update - We're conducting checks after the control plane upgrade and assess on when to continue with the upgrade.
Oct 5, 16:37 CEST
Identified - Sites start to come back online. We're monitoring the situation.
Oct 5, 16:13 CEST
Investigating - We're continuing to investigate the issue that causes the routes to fail.
Oct 5, 16:01 CEST
Update - We're seeing the issues with sites being down. We're investigating.
Oct 5, 15:57 CEST
Update - The control plane upgrade looks good. There could be intermittent deployment errors during the upgrade. Get in touch with our engineers and we'll be looking into the issue.
Oct 5, 15:44 CEST
Update - We found a small issue during the control plane upgrade which has been fixed - carrying on with the control plane upgrade.

As soon as we finish this part we'll carry on and update the compute nodes. We'll update here as soon as this will start.

During the compute node upgrade there might happen short downtimes due to the restart of the containers.
Oct 5, 14:51 CEST
Update - Control Plane update is still running - We'll update as soon as new information becomes available.
Oct 5, 13:38 CEST
Update - Control Plane update is currently running. We expect small <1min downtimes and will take action if needed.
Oct 5, 12:35 CEST
Identified - Pre-maintenance tasks have been completed sucessfully.

We plan to start the first part of the OpenShift upgrade momentarily.
Downtimes should be minimal (< 1 minute) . Our engineers are monitoring the situation closely and take action if needed.
Oct 5, 11:30 CEST
Investigating - amazee.io had planned to upgrade the ch1.lagoon cluster on October 11, 2018. However, due to persistent issues being experienced on the cluster, we have decided to trigger an emergency maintenance window, and begin upgrading the cluster immediately. We believe that this will bring stability to the cluster, and resolve the issues being experienced by a number of our customers.

Our team is currently polishing the plans for the upgrade, and will post a follow up as soon as we have solidified the strategy and timeline for the upgrade.

We are sorry for any inconvenience this may cause, and would like to assure our customers that we are doing everything within our power to keep downtime to an absolute minimum.

If you need any further information, our engineering team are available on Slack & Rocketchat. You are also welcome to reach out to us via email using support@amazee.io.
Oct 5, 11:19 CEST
Oct 4, 2018

No incidents reported.

Oct 3, 2018
Completed - The scheduled maintenance has been completed.
Oct 3, 02:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Oct 2, 22:01 CEST
Scheduled - We are conducting regular maintenance on our systems.
Oct 2, 17:52 CEST