status.amazee.io
Filter By Components

January 2017

[Scheduled] Emergency Maintenance
The scheduled maintenance has been completed.
Jan 18, 21:00 - Jan 19, 02:00 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jan 17, 21:00 - Jan 18, 02:00 UTC
[Scheduled] Additional maintenance for uk1-compact
The scheduled maintenance has been completed.
Jan 18, 01:15-01:29 UTC
Github service outage
Github issues resolved, our API is back to full speed ahead!
Jan 13, 16:34-17:04 UTC
API Unavailable
The hosting provider identified a memory-related maintenance issue and was in the process of deploying the fix as we contacted them. The API is again fully functional.
Jan 11, 01:02-01:17 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jan 10, 21:00 - Jan 11, 01:00 UTC
Partial Outage on API
All API services are restored and available again.
Jan 9, 11:19-11:49 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jan 3, 21:17 - Jan 4, 00:30 UTC

December 2016

Mitigation of CVE-2016-10045
PHPMailer verison 5.2.21 is released which solves CVE-2016-10045. Affected Customers have been informed directly. - https://github.com/PHPMailer/PHPMailer/releases
Dec 28, 08:52 - Dec 29, 11:18 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Dec 27, 21:00 - Dec 28, 03:00 UTC
Mitigation of DRUPAL-SA-PSA-2016-004
The Drupal Security Team just announced a new highly critical security announcement (23/25 points) for the 3rd Party Library PHPmailer: https://www.drupal.org/node/2839366 There is no Contrib or Core Module of Drupal directly affected, but multiple Contrib Modules of Drupal download the 3rd Party Library PHPmailer in order to send emails. Every website that uses PHPmailer is encouraged to update to the a newer version of PHPmailer then 5.2.18 (current available version is 5.2.19). As amazee.io cannot mitigate this issue from an infrastructure level, we informed all our clients which have PHPmailer 5.2.18 or earlier installed to update their codebase and with that fix their issue. If you did not receive such an information, our scripts did not find your website affected and there is probably nothing to do as of right now (even though amazee.io strongly suggests to keep all websites up to date at any time).
Dec 27, 02:24 UTC
Slack API issues
This incident has been resolved.
Dec 22, 15:59-22:05 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Dec 20, 21:00 - Dec 21, 00:00 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Dec 13, 21:00 - Dec 14, 00:00 UTC
uk1-compact connectivity
This incident has been resolved.
Dec 7, 18:43-20:53 UTC
Failing deployments due to drush alias issue
The issue has been resolved all drush aliases are now resolving as intended.
Dec 7, 08:55-09:07 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Dec 6, 21:00 - Dec 7, 00:00 UTC
GitHub Service Interruption
GitHub reports services are all now normal.
Dec 6, 20:32-20:58 UTC

November 2016

[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Nov 29, 21:00 - Nov 30, 00:00 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Nov 22, 21:00 - Nov 23, 00:00 UTC
uk1-compact scaling
uk1-compact has been scaled successfully
Nov 22, 23:01-23:41 UTC
Delayed deployments from GitHub
GitHub has reported that the webhook delivery delays are now resolved.
Nov 21, 16:52-17:42 UTC
Issues on dev1-compact
dev1-compact is back fully operational, we also noticed an issue with deploys to the server after it was fixed, and have resolved that as well.
Nov 17, 17:19-18:44 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Nov 15, 21:15 - Nov 16, 00:01 UTC
Limited availablility of sa1.compact
Connectivity to sa1.compact has been fully restored.
Nov 14, 14:49 - Nov 15, 06:24 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Nov 8, 21:01 - Nov 9, 00:00 UTC
Limited availablility of sa1.compact
The Level3 issues appear to be stabilized
Nov 3, 16:50-22:55 UTC
Issues on Deployment Infrastructure
We identified the issue and fixed all deployments should now run trough as expected. If a deployment is missing just restart the deployment again. If you still seeing issues with the deployments, don't hesitate to get in touch with us.
Nov 3, 08:09-08:57 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Nov 1, 20:00-23:01 UTC

October 2016

[Scheduled] Emergency Maintenance
The scheduled maintenance has been completed.
Oct 26, 20:00 - Oct 27, 00:00 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Oct 25, 20:00-23:00 UTC
Git webhooks not reliably triggering deploys
Github webhooks are working as expected.
Oct 21, 18:06 - Oct 22, 16:35 UTC
[Scheduled] Priority Maintenance - CVE-2016-5195
The scheduled maintenance has been completed.
Oct 21, 22:15 - Oct 22, 02:01 UTC
Github Webhooks don't trigger deplyoments
Accdording to our monitoring the webhooks are yielding positive results by now. Get in touch with us if you still face issues with github webhooks
Oct 21, 13:02-13:57 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Oct 18, 20:01-23:01 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Oct 11, 20:00-23:00 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Oct 4, 20:01-23:00 UTC

September 2016

US Provider maintenance
The maintenance has been finished.
Sep 22, 04:31-08:27 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Sep 20, 20:16-23:25 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Sep 13, 20:07-23:18 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Sep 13, 19:59 UTC
API Issues
The issue has been resolved.
Sep 9, 13:01-15:17 UTC
ZH1.Cluster Database Issues
Our monitoring and logs show that the issues with the zh1.cluster mariadb database cluster have been resolved with downgrading to an older version. We are in contact with the software provider to find a fix for this issue.
Sep 1, 13:35 - Sep 2, 06:40 UTC
Level 3 Connectivity Issues
Issues with Level 3 connectivity have been resolved.
Aug 30, 13:13 - Sep 1, 13:36 UTC

August 2016

[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Aug 23, 23:00 UTC
Load issues on zh1.compact
The issue has been resolved. server load and pageload times are back to normal.
Aug 23, 08:57-12:09 UTC
SA-CONTRIB-2016-047 - Infrastructure Hardening
The Drupal security team released a new version of the Drupal 7 Panels Contrib module which includes a security patch for a CRITICAL security issue: https://www.drupal.org/node/2785631 As every Drupal 7 site with Panels is affected and there are no mitigation actions for this security issue, we decided to roll out infrastructure hardening in order to reduce the attack vector for this security issue. This hardening is not able to completely protect Drupal 7 sites with Panels installed, but it makes it harder for an attacker to attack a site. The amazee.io team suggest everybody to update their panels version to 7.x-3.6 as soon as possible.
Aug 18, 00:49 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Aug 9, 20:15-23:00 UTC
Issues on us1.compact
Maintenance has been successfully finished.
Aug 9, 09:00-09:15 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Aug 2, 20:04 - Aug 3, 15:30 UTC

July 2016

[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jul 26, 20:07 - Jul 27, 03:30 UTC
Deployments failing due to upstream issues on git.drupal.org / github.com / bitbucket.org
We don't see any issues with pulling from git.drupal.org / github.com / bitbucket.org anymore.
Jul 18, 12:24 - Jul 26, 11:49 UTC
Issues on us1.compact
This incident has been resolved.
Jul 26, 09:28-11:49 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jul 19, 20:04-21:33 UTC
Issues on zh1.cluster
The Database Cluster is fully working and did not show any additional issues. A postmortem about this issue will be posted shortly
Jul 19, 14:21-17:43 UTC
Issues on us1.compact
We performed weekly maintenance on the us1.compact server. The maintenance window has ended successfully.
Jul 19, 09:35-10:49 UTC
CVE-2016-5385: Httpoxy Vulnerability
The patch is rolled out to all servers. There where a few downtimes of a couple of seconds for some sites. At this point every site hosted on amazee.io is completely secure against HTTPoxie / CVE-2016-5385. We suggest to update your Drupal 8 sites to version 8.1.7.
Jul 18, 14:32-15:32 UTC
[Scheduled] Information regarding remote code execution PSA-2016–001
We conducted a search through all sites hosted on amazee.io: - found one which had the module "webform_multifile" installed, this website has been patched on production - found multiple sites that have the coder module existing (remember it just needs to exist, not specifically being installed). As amazee.io infrastructure protects such attacks, there is no urgent need to update this module. We informed the clients that have these modules existing.
Jul 13, 16:18-16:55 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jul 12, 20:00 - Jul 13, 00:15 UTC
[Scheduled] uk1.compact Maintenance
The scaling operation has been completed.
Jul 8, 00:01-00:25 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jul 5, 20:00-23:45 UTC
Inter-continental transit issues
Our monitoring indicates this issue is now resolved.
Jul 1, 17:45-20:29 UTC

June 2016

[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jun 28, 21:03-23:49 UTC
Issues on dev1.compact
Issues related to the Solr Search Backend have been solved.
Jun 16, 17:21-20:33 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Jun 14, 20:00-22:14 UTC
Issues on us1.compact
Weekly maintenance has been performed.
Jun 14, 07:13-08:28 UTC
Issues on zh1.cluster
This incident has been resolved.
Jun 7, 22:48 - Jun 8, 05:44 UTC

May 2016

[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
May 31, 20:30-22:00 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
May 24, 20:03-21:06 UTC
Issues on nameservers
This incident has been resolved.
May 21, 16:10-17:20 UTC
[Scheduled] Regular Weekly Maintenance
Weekly maintenance is complete on all systems.
May 17, 20:00-21:35 UTC
Maintenenance on us1.compact
Maintenance is done. This incident has been resolved.
May 17, 09:21-09:28 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
May 10, 20:15 - May 11, 00:15 UTC
Issues on us1.compact
The issue has been solved. We had a short interruption in service with our upstream internet connection. Affected were only services running on us1.compact.
May 7, 07:57-08:04 UTC
[Scheduled] Regular Weekly Maintenance
Maintenance has been successful completed
May 3, 22:45-22:56 UTC

April 2016

Issues on zh1.cluster
This incident has been resolved.
Apr 28, 12:44-13:35 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Apr 26, 22:00 - Apr 27, 01:00 UTC
Issues on zh1.compact
This incident has been resolved.
Apr 24, 04:49-04:52 UTC
Issues on dev1.compact
This incident has been resolved.
Apr 20, 05:25-05:53 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Apr 19, 20:16-21:38 UTC
Issues on nameservers
This incident has been resolved.
Apr 14, 22:09-22:10 UTC
Issues on zh1.compact
This incident has been resolved.
Apr 14, 12:37-12:38 UTC
Issues on us1.compact
This incident has been resolved. Our engineers are monitoring the server closely and will act accordingly if there are further issues.
Apr 13, 09:00-09:23 UTC
Issues on attribute1.compact
This incident has been resolved.
Apr 12, 20:22-20:22 UTC
Issues on zh1.compact
This incident has been resolved.
Apr 12, 18:35-18:36 UTC
Issues on ublox1.compact
This incident has been resolved.
Apr 11, 11:28-11:34 UTC
Issues on attribute1.compact
This incident has been resolved.
Apr 7, 19:17-19:17 UTC
Issues on attribute1.compact
This incident has been resolved.
Apr 7, 13:11-13:12 UTC
Issues on attribute1.compact
This incident has been resolved.
Apr 7, 12:28-12:29 UTC
Issues on attribute1.compact
This incident has been resolved.
Apr 7, 10:09-10:10 UTC
Regular Weekly Maintenance
Maintenance has been completed.
Apr 5, 20:05-21:46 UTC
Issues on nameservers
This incident has been resolved.
Apr 5, 13:59-14:01 UTC
Issues on zh1.compact
This incident has been resolved.
Apr 3, 05:00-05:04 UTC

March 2016

Issues on attribute1.compact
This incident has been resolved.
Mar 31, 15:31-15:32 UTC
[Scheduled] Regular Weekly Maintenance
The scheduled maintenance has been completed.
Mar 30, 22:00 - Mar 31, 02:00 UTC
Issues on sa1.compact
The upstream provider had connectivity issues which had impact on our availability. Everything is back online according to our monitoring.
Mar 30, 18:51-18:58 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 28, 21:28-21:51 UTC
Issues on sa1.compact
We have been notified that the DDOS attack has been mitigated, all services are back to normal.
Mar 28, 06:56-16:58 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 28, 05:22-05:45 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 28, 02:16-02:50 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 28, 01:31-01:55 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 27, 23:31-23:51 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 27, 23:10-23:20 UTC
Issues on zh1.cluster
This incident has been resolved.
Mar 26, 13:43-13:44 UTC
Issues on jenkins
This incident has been resolved.
Mar 22, 20:49-20:55 UTC
Issues on jenkins
This incident has been resolved.
Mar 22, 20:46-20:47 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 22, 17:42-17:45 UTC
Issues on nameservers
This incident has been resolved.
Mar 22, 12:42-13:01 UTC
Issues on sa1.compact
This incident has been resolved.
Mar 20, 20:06-20:10 UTC
DNS Issues on dev1.compact and Jenkins
This incident has been resolved.
Mar 18, 11:00-12:40 UTC
[Scheduled] Regular Weekly Maintenance
Maintenance completed, everything happy.
Mar 15, 21:09-23:35 UTC