amuck-landowner

ChicagoVPS / CVPS Hacked. New SolusVM exploit? (Content Restored) [PT. 1/2]

Status
Not open for further replies.
V

vkimball

New Member
I had 3 of my 5 vps' affected by the CVPS incident.

2 of them (in Atlanta) were back up on Thursday with a clean install of Ubuntu 12.04.1 and no nameservers defined.  Easy enough to change my root password, configure nameservers and update to current.

The other one (in LA) was back up on Friday with a clean CentOS 5.8 install rather than Ubuntu 12.04.  Unfortunately, I can't reinstall my vps because WHMCS doesn't allow that function.  Guess I'll have to open a ticket and wait.

I don't really care about backups because I was only using them as test environments and all the important data was stored elsewhere.
 
MannDude

MannDude

Just a dude
vpsBoard Founder
Moderator
jer said:
Matthew || Staff  Saturday, June 22nd, 2013 (15:08)

At this time virtually all customers are back online; some with all original files in tact and others with new containers. File restoration is possible, though must be done manually by our staff. Right now we only have 2 more nodes to fix, and once that process is completed we are going to begin working through the long list of requests for file restoration from our backups. That process will take 3-4 days before all requests have been fulfilled.


We are happy to restore your files, though if its easier and quicker for you to restore the files yourself from your own backups we recommend going in that direction.


You now have full control over your VPS from WHMCS (you can stop/start/restart/reset root PW/reload it, etc).


Thank you again for your business and support.


---------------


Matthew


Support Guru

Jer S || Client  Saturday, June 22nd, 2013 (09:42)

Hey folks, still down. No ping.


I do not need a restore. I've looked at my rows in the leaked database, my container parameters don't seem broken.


----------------------------

Chris's response makes it sound like I want a restore. I don't. I don't know how to type it clearer.

So if you go in the Client Area to Services > Manage My VPS   - - it's broken.

Servers still down.
Click to expand...
Matthew, I am 98% sure is indeed Chris. Just an alias.
 
mikho

mikho

Not to be taken seriously, ever!
Aldryic C said:
Food for thought: SolusVM has a backup system included. However, information is stored much the same way as node information is - and if the attacker could wipe the nodes, there was nothing stopping him from wiping even remote backups as well if they were tied into Solus.


Sure, there's always the chance that they used different software, or wrote their own scripts for backups. But I wouldn't wager on that.
Click to expand...
I guess this is what happened to Virpus back in 2011 when 19 of their nodes where taken out as in complete wipe with no backups.
 
S

srichter

New Member
I'm not sure why they keep responding to the tickets with canned responses (well I assume they hope we give up). They say that it's possible for them to restore from backups, but then when you request that you just get a canned response. Why not say "We have added you to the queue" and mark the ticket "On Hold" or "In Progress" instead of just sending out another canned email? How many times am I going to have to request they restore from the backup?

At this time virtually all customers are back online; some with all original files in tact and others with new containers. File restoration is possible, though must be done manually by our staff. Right now we only have 2 more nodes to fix, and once that process is completed we are going to begin working through the long list of requests for file restoration from our backups. That process will take 3-4 days before all requests have been fulfilled.

We are happy to restore your files, though if its easier and quicker for you to restore the files yourself from your own backups we recommend going in that direction.

You now have full control over your VPS from WHMCS (you can stop/start/restart/reset root PW/reload it, etc).

Thank you again for your business and support.
Click to expand...
 
D

drvelocity

New Member
srichter said:
I'm not sure why they keep responding to the tickets with canned responses (well I assume they hope we give up). They say that it's possible for them to restore from backups, but then when you request that you just get a canned response. Why not say "We have added you to the queue" and mark the ticket "On Hold" or "In Progress" instead of just sending out another canned email? How many times am I going to have to request they restore from the backup?
Click to expand...
Amen - the whole thing is so hilariously ridiculous.  The worst possible result one could ever expect in a situation like this.  Anyone who actually gets a backup restored please make sure to let us know here, because my best guess is that it's all total BS.  
 
L

Lanarchy

New Member
upsetcvps said:
http://www.facebook.com/l.php?u=https%3A%2F%2Fbilling.chicagovps.net%2Fclientarea.php%3Faction%3Dchangepw&h=[omitted]&s=1

Can anyone explain to me why the "change your password" link in their latest e-mail goes through facebook...?
Click to expand...
He posted to FB first, then just copy/paste to the email, including the FB outgoing link.

My stats, I have not opened a ticket asking for any restores.

NY - good, original data ... an hour ago randomly messed up the firewall but I did csf -r and all is well again. Probably coincidence, but who knows at this point.

CHI - good, original data

LA - good, original data

ATL - one fresh install and accessible, one unknown and serial console says 'console configuration not found' and will not boot.

However, for safety, once we get a real control panel, I will reinstall all of them fresh. Just in case any passwords were saved, or any files inserted (which noone has mentioned yet, but I assume is a possibility)
 
Last edited by a moderator:
S

srichter

New Member
I replied to their canned response with

I understand you may have other things to work on before restoring from the backup, but this is my request that you do so. Please do not mark this ticket as answered, please insert me into the queue for my data to be restored as soon as it is possible for you to do so.

"File restoration is possible, though must be done manually by our staff." - Please do so

"We are happy to restore your files" - Please do so

Thank You
Click to expand...
And they replied

Steve,

You got it.

---------------
Matthew
Support Guru
Click to expand...
And changed the status of the ticket to "Restoration."

That's a good sign!
 
V

vkimball

New Member
vkimball said:
The other one (in LA) was back up on Friday with a clean CentOS 5.8 install rather than Ubuntu 12.04.  Unfortunately, I can't reinstall my vps because WHMCS doesn't allow that function.  Guess I'll have to open a ticket and wait.
Click to expand...
Well, I'm happy to report that my LA vps was just reinstalled as Ubuntu 12.04.
 
upsetcvps

upsetcvps

New Member
Can anyone that has a fresh debian (squeeze) install on a cvps openvz container, post their ssh server public key (or fingerprint)?  The fingerprint is what you see when you connect to a server for the first time to verify its identity.  You can obtain the fingerprint by running the command: 

ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key
 
Last edited by a moderator:
T

Tactical

Where is the beer!
I like to throw this out there, all the griping about canned responses. How would you respond if you had 2000 ppl saying the same thing? The exact same way. I wish everyone the best of luck.  The lesson is to back up your own data. Peace out
 
upsetcvps

upsetcvps

New Member
SgtZinn said:
I like to throw this out there, all the griping about canned responses. How would you respond if you had 2000 ppl saying the same thing? The exact same way. I wish everyone the best of luck.  The lesson is to back up your own data. Peace out
Click to expand...
They should display the canned response on the page where people submit tickets.  That way if the canned response actual answers your question you don't even create a ticket!

But I imagine the griping is because the canned response doesn't actually address the issue in the ticket.  Which means you have to open the ticket again until someone bothers to actually read what you typed.  
 
T

Tactical

Where is the beer!
I see your point. Its a big mess but there main priority is get the nodes up then try to work off the tickets. So that is probably y ppl are getting canned responses. It just takes time. It could take weeks to get it all worked out, but I'm no expert.
 
Hugohp

Hugohp

New Member
I just got this email from them. 

Hello,

Unfortunately backups for your container from our master backup repository are not available. If you utilized our free Central Backup feature to create a restore point for your service we can backup from that data. If you did not utilize that free service we do not have backups and will be unable to restore any of your data. 


---------
Luc Ayotte
ChicagoVPS Support Tech
[email protected]
Click to expand...
No backup for me......
 
H

helobye

New Member
Anyone else have this problem? CVPS have configured an empty Ubuntu 12.04 container for me on LA-18, but the VM is unable to access the internet, and I can't SSH in. I added a few DNS servers and changed the hostname (using the Serial Console) to no avail. Iptables is empty.
 
Last edited by a moderator:
Status
Not open for further replies.
Top
amuck-landowner