amuck-landowner

ChicagoVPS / CVPS Hacked. New SolusVM exploit? (Content Restored) [PT. 1/2]

Status
Not open for further replies.
D

drvelocity

New Member
My account was just restored, and the data looks like it's from June 10 - (I'm on a NY server).  I have to say I thought this was going to drag on for more than a week - they just made my weekend.  I have to eat my previous words, they came through for me on this one, I very sincerely hope that everyone else affected gets all fixed up as well.  Good luck all!  And don't forget to back up! ;)
 
H

helobye

New Member
Just got this notice of no backups, node LA-18: 

Hello,

Unfortunately backups for your container from our master backup repository are not available. If you utilized our free Central Backup feature to create a restore point for your service we can backup from that data. If you did not utilize that free service we do not have backups and will be unable to restore any of your data. 


---------
Luc Ayotte
ChicagoVPS Support Tech
[email protected]
Click to expand...
 
Last edited by a moderator:
MannDude

MannDude

Just a dude
vpsBoard Founder
Moderator
It's somewhat interesting seeing the contrast between the updates in this thread and the updates in the LET thread.
 
W

whatever

New Member
they emailed me today

Hello,

Unfortunately backups for your container from our master backup repository are not available. If you utilized our free Central Backup feature to create a restore point for your service we can backup from that data. If you did not utilize that free service we do not have backups and will be unable to restore any of your data. 


---------
Luc Ayotte
ChicagoVPS Support Tech
[email protected]
Click to expand...
my node in LA,I think they don't care for Mexico, terrible situation (MX customer as Hugohp)
 
H

helobye

New Member
whatever said:
they emailed me today

my node in LA,I think they don't care for Mexico, terrible situation (MX customer as Hugohp)
Click to expand...
Sorry to hear your node was unable to be restored as well. If your containers IP address is in the 198.46.137.x range it's node LA-18 like mine.
 
W

whatever

New Member
helobye said:
Sorry to hear your node was unable to be restored as well. If your containers IP address is in the 198.46.137.x range it's node LA-18 like mine.
Click to expand...
Yes, I am in the same range, I had a hope from them when I saw "We are happy to restore your files..", but I 'll have to do everything again. Long nights since today, 
 
drmike

drmike

100% Tier-1 Gogent
Aldryic C said:
What really should concern people is the database leak, though.  @buffalooed could confirm this... but wasn't the DB leak dated at least a day or more before the actual attack?  The guy could've grabbed any number of VPS dumps in the meantime before tearing everything up.
Click to expand...
The database for SolusVM was borrowed on the 17th.

RamNode was hit by this Solus issue a day or more before this.

RamNode got taken down Sunday morning 9AM or earlier Eastern time on Sunday, June 16, 2013.

ChicagoVPS noticed their hack and node damage around 2AM Eastern time on Monday, June 17th, 2013.  

That means the exploit existed and was known for some 17 hours between these events. 
 
drmike

drmike

100% Tier-1 Gogent
MannDude said:
They've got 4 backup servers listed in Solus to handle backing up 109~ nodes, minus the Atlanta location. By the sounds of it, there is more than just Atlanta that wasn't backed up. LA, Chicago and Buffalo appear to be the only physical locations with backup servers
Click to expand...
 

Yeah well, they aren't backing everything up.

Backing all this up across the internet = mega slow process in mass.

I'm pretty sure if you are on one of the following nodes, your data is gone for good:

 

+--------+-----------+-----------------------------+

| nodeid | name      | hostname                    | number of vservers on nod |

|      1 | localhost | manage.chicagovps.net       |

|     35 | chi22     | chi-vps22.chicagovps.net    | 10 vservers

|     21 | chi10     | chi-vps10.chicagovps.net    | 23 vservers

|     25 | chi13     | chi-vps13.chicagovps.net    | 11 vservers

|     31 | chi18     | chi-vps18.chicagovps.net    | 17 vservers

|     37 | chi24     | chi-vps24.chicagovps.net    | 11 vservers

|     39 | chi23     | chi-vps23.chicagovps.net    | 15 vservers

|     42 | chi27     | chi-vps27.chicagovps.net    | 16 vservers

|     79 | chissd1   | chi-ssd-vps1.chicagovps.net | 59 vservers

|     48 | chi32     | chi-vps32.chicagovps.net    |  13 vservers

|     49 | chi33     | chi-vps33.chicagovps.net    |  0 vservers

|     57 | chi40     | chi-vps40.chicagovps.net    | 23 vservers

|     65 | chi47     | chi-vps47.chicagovps.net    | 26 vservers

|     68 | chi50     | chi-vps50.chicagovps.net    | 29 vservers

|     76 | chi51     | chi-vps51.chicagovps.net    | 21 vservers

|     80 | chi53     | chi-vps53.chicagovps.net    | 80 vservers

|    109 | atl1      | atl-vps1.chicagovps.net     | 161 vservers

|    110 | atl2      | atl-vps2.chicagovps.net     | 183 vservers

|    128 | atl3      | atl-vps3.chicagovps.net     |  56 vservers

|    131 | atl4      | atl-vps4.chicagovps.net     | 122 vservers

|    133 | atl5      | atl-vps5.chicagovps.net     | 92 vservers

|    138 | atl6      | atl-vps6.chicagovps.net     | 109 vservers

|    148 | nj1       | nj-vps1.chicagovps.net      | 13 vservers

|    149 | dfw1      | dfw-vps1.chicagovps.net     | 5 vservers

|    150 | njkvm1    | nj-kvm-vps1                 | 3 vservers

|    151 | chi70     | chi-vps70.chicagovps.net    | 31 vservers

+--------+-----------+-----------------------------+

 

Simply said, those 26 servers were not configured for FTP backups via Solus.

 

There are roughly 104 nodes that were live.  26 un-backedup servers represents an even 25% of their servers that weren't being backed up.
 
S

Swift

New Member
At this time virtually all customers are back online; some with all original files in tact and others with new containers. File restoration is possible, though must be done manually by our staff. Right now we only have 2 more nodes to fix, and once that process is completed we are going to begin working through the long list of requests for file restoration from our backups. That process will take 3-4 days before all requests have been fulfilled. 
We are happy to restore your files, though if its easier and quicker for you to restore the files yourself from your own backups we recommend going in that direction.

You now have full control over your VPS from WHMCS (you can stop/start/restart/reset root PW/reload it, etc). 

Thank you again for your business and support.

---------------

Matthew

Support Guru
Click to expand...
 I hate myself for not having a recent backup.
 
jer

jer

New Member
Josh Aborad || Staff  Saturday, June 22nd, 2013 (20:34)

Jer,
We will investigate this issue for you and report back when we have more information for you. Thank you for your patience.
Josh
Support Guru

 

 

Yada yada... just don't know what to say.
 
drmike

drmike

100% Tier-1 Gogent
srichter said:
My VPS didn't go down until ~3AM EST on the 18th
Click to expand...

That's very strange.  Know I saw others say Tuesday they went down.  Pretty sure that was the admins trying to fix something and probably declaring the node unsalvageable, necessitating a full server rebuild.
 
drmike

drmike

100% Tier-1 Gogent
srichter said:
The 18th was Tuesday
Click to expand...
Yes, indeed.  Odd to have nodes go down 24 hours+ after the attack.  Wondering if things were left running on some of these servers from the attack.

I'd be mighty concerned if I were on a node that did this on Tuesday. 
 
Status
Not open for further replies.
Top
amuck-landowner