Stories
Slash Boxes
Comments

SoylentNews is people

Meta
posted by martyb on Sunday July 24 2016, @12:36PM   Printer-friendly
from the ups-and-downs dept.

We just learned that our VM provider, Linode, had to perform some emergency reboots. Three of our servers have already been taken care of, but more are still to come. This led to our site being unavailable for a period of approximately an hour. Here is the reboot schedule:

Identified - Linode has received a Xen Security Advisory (XSA) that requires us to perform updates to our legacy Xen host servers. In order to apply the updates, hosts and the Linodes running on them must be rebooted. The XSAs will be publicly released by the Xen project team on July 26th. We must complete this maintenance before then.

Here's the schedule and status:

Server Purpose Maintenance Schedule (UTC)
lithium Development Completed
magnesium Frontend Proxy Completed
sodium Frontend Proxy Completed
fluorine Production Cluster Completed
helium Production Cluster Completed
hydrogen Production Cluster Completed
neon Production Cluster Completed
beryllium Services Cluster Completed
boron Services Cluster Completed

We apologize for any inconvenience.

[Update: It appears the second round of reboots has completed successfully, and, thanks to the advance notice, the site stayed up throughout. We anticipate that the site will still continue to operate normally through the last-scheduled reboot. Many thanks for your understanding and patience.]

[Update #2: We are taking advantage of a free offer from Linode, our hosting provider, to convert our VPSs (Virtual Private Servers) from Xen to KVM. The rebooting was required to repair a Xen vulnerability. As a bonus, the Xen to KVM conversion gives us a free upgrade to twice as much memory. The additional memory will provide much-needed additional headroom on our servers and possibly provide a performance improvement. Thanks to our redundancy, the changes should not be noticeable when we reboot/upgrade, except for the IRC and e-mail servers as they are single-hosted.]

[Update #3: Thanks to the tireless efforts of paulej72 well into the wee hours of this morning with able assistance by audioguy in straightening out some IP issues as well as Deucalion and TheMightyBuzzard providing guidance and support, all but two of our Xen servers have been upgraded to KVM. This free upgrade doubled the amount of memory available to our VMs, giving us some much-needed headroom. That leaves beryllium (IRC and email) and boron (DNS, Hesiod name service) as the two servers that have not been upgraded yet. Date/time is TBD.]

[Update #4: Boron will be reconfigured shortly, and then Bberyllium after that. Plan on an hour or two, though, obviously, we'll try to keep the downtime to a minimum!]

[Update #5: Boron's second upgrade for the ram sat in the queue for several hours, so Beryllium had to wait until paulej72 got up and finished it this morning (0830 EDT)]

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by isostatic on Friday July 22 2016, @07:44AM

    by isostatic (365) on Friday July 22 2016, @07:44AM (#378383) Journal

    Linode told me too, on Wednesday 1230 GMT

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2