We have identified an issue on the Atlas server that requires immediately attention and will initiate downtime shortly. Downtime should be less than 30 minutes and there should be no other adverse effects of this maintenance.
This maintenance has been completed and service accessibility and optimum speeds should be restored on the Atlas server at this time.
The Goliath server has crashed this morning. We are looking in to the cause and working towards a resolution at this time. We will bring Goliath back online as soon as possible, however we do not yet have an ETA for resolution. Further updates will be posted here as they become available.
Update 9:30AM CDT: Goliath is running a Filesystems Check (FSCK) and should be back online within 15-30 minutes.
Update 9:36AM CDT: Goliath is back online at this time and services should again be accessible.
Final Update 9:52AM CDT: We do not have a specific cause of the crash however we can attribute the most likely cause to an I/O error. As such, we are tweaking configurations to help ensure this does not occur again, and enabling additional logging to monitor the server more closely for some time (all RAID arrays are in tact and optimal, and a FSCK was completed successfully, as well). We apologize for the downtime and would like to thank everyone for their patience this morning.
Start Date: 06/15/2014
Start Time: 09:00 PM Central Time
Estimated End Date: 06/16/2014
Estimated End Time: 02:00 AM Central Time
Service: OnApp Platform Upgrade
Type of Work: Software Upgrade and Testing
Impact of Work: Intermittent service availability of VPS servers during upgrade and testing during the maintenance window
This notification is to inform you that on Sunday, June 15th, 2014, beginning at 09:00 PM Central Time, with an estimated ending on Monday, June 16th, 2014, at 2:00 AM Central Time, the OnApp Platform will have intermittent inaccessibility during a planned software upgrade and necessary testing. All support tickets will be answered as usual. We do ask that you refrain from submitting a ticket regarding your VPS during this maintenance window as any issue(s) you encounter are likely a direct result of this maintenance.
Update #1 - 9:58 PM Central Time, 06/15/2014
The start of the maintenance has been delayed until 10 PM Central Time. We still hope to have the maintenance fully completed by the 2 AM Central Time estimated ending.
Update #2 - 2:17 PM Central Time, 06/16/2014
We are wrapping up some maintenance work on the backend, but otherwise the maintenance has been completed successfully. Most VPS experienced less than 5 minutes of downtime (if any) and there were only a handful that experienced a slightly longer downtime, so we're thankful this maintenance went quite smoothly overall.
We will need to perform about 1 more hour of maintenance on the platform at 11 PM Central Time 06/16/2014. This is a pretty minor maintenance operation and we don't expect more than 5 minutes of downtime (if any) for all VPS on the OnApp platform.
Notice of Scheduled Maintenance Window
Service Affected: All VPS Servers
Maintenance Type: Software upgrade
Impact of Work: Minimal downtime, but intermittent slow speeds are likely throughout the maintenance window. Server reboots may be necessary during this maintenance window. Do not plan for the OnApp Control Panel to be available during this maintenance window.
Start Date: 03/30/2014
Start Time: 11:00 PM CDT (GMT -5)
End Date: 03/31/2014
End Time: 04:00 AM CDT (GMT-5)
We are aware of an issue with OnApp currently that is causing some VM's to be offline and not allowing them to start. The issue is being investigated by OnApp support and ourselves and we hope to have a resolution soon.
We apologize for any inconvenience and I'd like to thank you in advance for your patience.
UPDATE @ 11:34 AM CST
Most of the affected VM's have been migrated to other hypervisors and are back online at this time. The root cause of this issue is still being investigated by the OnApp developers.
UPDATE @ 4:30 PM CST
HV8 is being rebooted at this time.
UPDATE @ 4:41 PM CST
All affected VM's are back online. OnApp is still investigating the root cause of the issue.