status.amazee.io
All Systems Operational

About This Site

Issues on amazee.io but nothing shown here?
Let us know on RocketChat or support@amazee.io, and we'll help you right away.

Documentation? Find everything at docs.amazee.io
What is amazee.io?

General Operational
API Operational
Deployment Infrastructure Operational
Nameservers Operational
dev1.compact Operational
Switzerland Operational
zh1.single Operational
zh2.single Operational
zh1.compact Operational
zh2.cluster Operational
zh2.compact Operational
ch1.lagoon ? Operational
USA Operational
us1.lagoon Operational
South Africa Operational
sa1.compact Operational
United Kingdom Operational
uk1.compact Operational
uk2.compact Operational
Germany Operational
de1.compact Operational
Finland Operational
fi1.compact Operational
Japan Operational
jp1.compact Operational
on-premise servers Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Scheduled Maintenance
Regular Weekly Maintenance - US Mar 19, 14:00-18:00 CET
We are conducting regular maintenance on our systems.
Posted on Mar 11, 15:00 CET
jp1-compact: Ubuntu LTS upgrade Mar 19, 15:00-19:00 CET
As the operating system of the legacy Drupal hosting will run out of support soon we will perform a operating system upgrade from Ubuntu 14.04 LTS to Ubuntu 16.04 LTS.

We will conduct maintenance on 2019-03-19 starting 15:00 CET / 23:00 JST and will last for about 4 hours. Availability of the sites during the maintenance may be limited.

https://www.worldtimebuddy.com/?qm=1&lid=2657896,1850147&h=2657896&date=2019-3-19&sln=15-19.5
Posted on Mar 13, 11:13 CET
Regular Weekly Maintenance - Europe Mar 19, 22:00 - Mar 20, 02:00 CET
We are conducting regular maintenance on our systems.
Posted on Mar 11, 23:00 CET
fi1-compact: Ubuntu LTS upgrade Mar 20, 22:00 - Mar 21, 02:00 CET
As the operating system of the legacy Drupal hosting will run out of support soon we will perform a operating system upgrade from Ubuntu 14.04 LTS to Ubuntu 16.04 LTS.

We will conduct maintenance on 2019-03-20 starting 22:00 CET and will last for about 4 hours. Availability of the sites during the maintenance may be limited.
Posted on Mar 13, 14:29 CET
de1-compact: Ubuntu LTS upgrade Mar 21, 22:00 - Mar 22, 02:00 CET
As the operating system of the legacy Drupal hosting will run out of support soon we will perform a operating system upgrade from Ubuntu 14.04 LTS to Ubuntu 16.04 LTS.

We will conduct maintenance on 2019-03-21 starting 22:00 CET and will last for about 4 hours. Availability of the sites during the maintenance may be limited.
Posted on Mar 13, 14:29 CET
sa1-compact: Ubuntu LTS upgrade Mar 21, 22:00 - Mar 22, 02:00 CET
As the operating system of the legacy Drupal hosting will run out of support soon we will perform a operating system upgrade from Ubuntu 14.04 LTS to Ubuntu 16.04 LTS.

We will conduct maintenance on 2019-03-21 starting 22:00 CET and will last for about 4 hours. Availability of the sites during the maintenance may be limited.
Posted on Mar 13, 14:29 CET
Regular Weekly Maintenance - US Mar 26, 14:00-18:00 CET
We are conducting regular maintenance on our systems.
Posted on Mar 18, 15:00 CET
uk1-compact / uk2-compact - Ubuntu LTS upgrade Mar 26, 22:00 - Mar 27, 02:00 CET
As the operating system of the legacy Drupal hosting will run out of support soon we will perform a operating system upgrade from Ubuntu 14.04 LTS to Ubuntu 16.04 LTS.

We will conduct maintenance on 2019-03-26 starting 22:00 CET and will last for about 4 hours. Availability of the sites during the maintenance may be limited.
Posted on Mar 13, 14:30 CET
zh1-compact / zh2-compact - Ubuntu LTS upgrade Mar 27, 22:00 - Mar 28, 02:00 CET
As the operating system of the legacy Drupal hosting will run out of support soon we will perform a operating system upgrade from Ubuntu 14.04 LTS to Ubuntu 16.04 LTS.

We will conduct maintenance on 2019-03-27 starting 22:00 CET and will last for about 4 hours. Availability of the sites during the maintenance may be limited.
Posted on Mar 13, 14:30 CET
Past Incidents
Mar 18, 2019

No incidents reported today.

Mar 17, 2019

No incidents reported.

Mar 16, 2019

No incidents reported.

Mar 15, 2019

No incidents reported.

Mar 14, 2019
ch1-lagoon : Galera Database Issues
Resolved - This incident has been resolved.
Mar 14, 16:29 CET
Monitoring - A fix has been implemented and we are monitoring the results.
Mar 14, 16:20 CET
Identified - The issue has been identified and a fix is being implemented.
Mar 14, 15:58 CET
Investigating - We are currently investigating this issue.
Mar 14, 15:50 CET
uk1-compact : High Load
Resolved - This incident has been resolved.
Mar 14, 11:32 CET
Investigating - We are currently investigating this issue.
Mar 14, 11:26 CET
Mar 13, 2019
Regular Weekly Maintenance - Europe
Completed - The scheduled maintenance has been completed.
Mar 13, 02:00 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 12, 22:00 CET
Scheduled - We are conducting regular maintenance on our systems.
Mar 4, 23:00 CET
Mar 12, 2019
dev1-compact: Ubuntu LTS upgrade
Completed - The scheduled maintenance has been completed.
Mar 12, 23:00 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 12, 19:00 CET
Scheduled - As the operating system of the legacy drupal hosting will run out of support soon we will perform a operating system upgrade from Ubuntu 14.04 LTS to Ubuntu 16.04 LTS.

We will conduct maintenance on 2019-03-12 starting 19:00 CET and will last for about 4 hours.
Mar 7, 14:52 CET
Regular Weekly Maintenance - US
Completed - The scheduled maintenance has been completed.
Mar 12, 15:52 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 12, 14:00 CET
Scheduled - We are conducting regular maintenance on our systems.
Mar 4, 15:00 CET
us1-lagoon : creation of new environments
Resolved - This incident has been resolved.
Mar 12, 15:52 CET
Identified - We're seeing issues on us1-lagoon with newly created environments. Our engineers have identified the issue and are working on it.
Existing environments are not affected
Mar 12, 14:42 CET
Connectivity Issues for ch1.lagoon
Resolved - This incident has been resolved.
Mar 12, 15:52 CET
Monitoring - All sites have recovered, we're monitoring the situation.
Mar 12, 15:41 CET
Identified - We have identified the issues being connected to a specific cluster node. We're removing the faulty node from the cluster.
Mar 12, 15:36 CET
Investigating - We are currently investigating this issue.
Mar 12, 15:31 CET
Mar 11, 2019

No incidents reported.

Mar 10, 2019

No incidents reported.

Mar 9, 2019

No incidents reported.

Mar 8, 2019

No incidents reported.

Mar 7, 2019
Protecting unpatched sites from exploits - SA-CORE-2019-003
Resolved - This incident has been resolved.
Mar 7, 15:01 CET
Monitoring - Thank you to all the customers that were able to patch their sites quickly in response to PSA-2019-02-22.

We have disabled modules on sites that we have found to be vulnerable however we were not able to cover all sites for a number of reasons and have notified customers of the issues.

We still strongly recommend that customers check their sites and patch as soon as possible and will continue to monitor this incident until we can confirm all sites are mitigated.
Feb 24, 21:29 CET
Update - We are continuing to work through sites that might be vulnerable to the exploits, in the meantime we strongly recommend that customers patch their sites or disable the modules if you are not able to patch immediately.
Feb 23, 06:30 CET
Identified - There are public exploits now available for SA-CORE-2019-003 -- https://www.drupal.org/psa-2019-02-22

This only applies to your site if:

The site has the Drupal 8 core RESTful Web Services (rest) module enabled.
- OR -
The site has another web services module enabled, like JSON:API in Drupal 8, or Services or RESTful Web Services in Drupal 7, or custom code that allows entity updates via non-form sources.


In response, amazee.io have made the decision to disable modules on unpatched sites to protect them from exploits.

We are planning to disable all modules on site that have yet to apply the latest patches. This change will be rolled out at 4:00 UTC to allow some time to patch sites. If you would like to opt out of this change please contact us as soon as possible.

We understand this may break functionality on some sites that depend on these modules but believe it is far better to deal with a broken site than a hacked one.

If you find the modules have been disabled we recommend you follow the mitigation instructions on the PSA before re-enabling the modules.

If you are using Drupal 8.6.x, upgrade to Drupal 8.6.10.
If you are using Drupal 8.5.x or earlier, upgrade to Drupal 8.5.11.
Be sure to install any available security updates for contributed projects after updating Drupal core.
Feb 23, 03:55 CET
uk1-compact load issues
Resolved - This incident has been resolved.
Mar 7, 14:49 CET
Investigating - We are currently investigating this issue.
Mar 5, 15:08 CET
Mar 6, 2019
Regular Weekly Maintenance - Europe
Completed - The scheduled maintenance has been completed.
Mar 6, 02:01 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 5, 22:00 CET
Scheduled - We are conducting regular maintenance on our systems.
Feb 25, 23:00 CET
Mar 5, 2019
Regular Weekly Maintenance - US
Completed - The scheduled maintenance has been completed.
Mar 5, 18:00 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 5, 14:00 CET
Scheduled - We are conducting regular maintenance on our systems.
Feb 25, 15:00 CET
ch1-lagoon : Availability issues on a subset of sites
Resolved - This incident has been resolved.
Mar 5, 14:38 CET
Identified - All affected sites are back online - One of the compute nodes seems to be affected. We moved the failing containers off this compute node and will put a fix in place in the afternoon.
Mar 5, 11:45 CET
Investigating - We're currently investigating an issue that only affects a small set of sites hosted on ch1-lagoon.
We already put measures in place to make sure all affected sites come back online momentarily.
Mar 5, 11:34 CET
Delayed lagoon deployments
Resolved - Issue was due to an upstream registry that was not responding, all services are back to normal now.
Mar 5, 05:54 CET
Investigating - Lagoon builds are delayed from what appears to be an unavailable RedHat registry.
Mar 5, 01:08 CET
Mar 4, 2019

No incidents reported.

← Incident History Powered by Statuspage