# GreenValueHost forced password reset - Security breach?



## MannDude

> Hello,
> 
> You might have received an email about the client area password reset. We are currently investigating the issue and will update you on what is happening later. Please do not open a support ticket about this issue.
> 
> Thank You
> 
> GreenValueHost Team



Unsure if anyone here is actually a customer, I know they have an active following on LowEndTalk however. The past few hours people over there have been mentioning their passwords being reset without their doing, looks GreenValueHost has officially released a statement via email.

If you're a customer I'd say it's probably advisable to make sure anything else in your name that uses the same password you were using is updated and changed. Never _ever_ wise to use the same password in two places.

They've not said what the issue is but if they have forced a password reset then you can only assume it was in response to something. Considering they're "investigating"  it and not stating it was a routine security procedure it's probably safe to assume something bad has happened.


----------



## HH-Josh

I've also received this email this morning at 0957 AM (GMT). I'm not even a customer with them and never have been so I'm not entirely sure where they're got my email from and why I've received the email.


Be interesting to hear why and how when they finally get round to admitting if "something went wrong".


----------



## rds100

@HL-Josh check the headers? Where did the email come from?


----------



## drmike

They use Mandrill to send emails BTW.

Curious about never being a customer.  Perhaps some other reason to be in their database?  Ticketing?  Sale lead?

Appears 6 emails were sent to each customer/email address.


----------



## MannDude

drmike said:


> Appears 6 emails were sent to each customer/email address.


At six different times or all at once?


----------



## drmike

MannDude said:


> At six different times or all at once?


Hard to say since these would get queued by Mandrill and may blow time gaps away or elongate artificially.

Know I heard someone say they just received one of those PW reset emails in past hour... About 6 hours into this by my estimate.


----------



## HH-Josh

drmike said:


> They use Mandrill to send emails BTW.
> 
> 
> Curious about never being a customer. Perhaps some other reason to be in their database? Ticketing? Sale lead?
> 
> 
> Appears 6 emails were sent to each customer/email address.


I've had absolutely no communication with this company. I never have even been as close to a potential client of GVH.


Going to drop them an email and find out why I'm listed in their database.


----------



## mtwiscool

drmike said:


> Hard to say since these would get queued by Mandrill and may blow time gaps away or elongate artificially.
> 
> Know I heard someone say they just received one of those PW reset emails in past hour... About 6 hours into this by my estimate.


i only received one email.


----------



## drmike

mtwiscool said:


> i only received one email.


Lucky you  There may be more emails..  Again, no one is sure and gets goofier (like person above with no relationship to GVH).


----------



## WebSearchingPro

It appears that it was accidental, I had my password reset then a second email shortly after saying its being looked into. I believe the investigation is into the person who reset the passwords.

To be fair though with all the trolling everyone is doing to GVH i.e. people spamming GVH threads and bumping everything thats ever existed on *that other forum,* it wouldn't surprise me if someone on the inside did it to make things look worse.


----------



## MartinD

You can't accidentally send an email to all customers telling them to reset their password.

You can't accidentally trip up and end up in the arms of a prostitute and claim you were only getting a hug.


----------



## drmike

Hard to say what happened, cause they don't know 

I know from talking with GVH folks overnight, that no one with credentials did anything to generate said emails about passwords.  Obviously, someone over there ought to be preserving all logs and manually going over things looking for the event(s).  Imagine later this morning that will happen.

I won't comment on that other website, beyond saying some other well known folks were tired of incessant GVH threads and went on a GVH thread creation of their own.   Like 80% of LET homepage was GVH threads.

Kossen exercised his powers and ban hammered Oktay and Spencer.   He also gave GVH a soft ban with no offers/coupons/shilling until June 1.

... and probably around that time... this email problem materialized...


----------



## rds100

MartinD said:


> You can't accidentally send an email to all customers telling them to reset their password.
> 
> You can't accidentally trip up and end up in the arms of a prostitute and claim you were only getting a hug.


There is a setting in WHMCS which says how to store the passwords in the database. I imagine if someone accidently changes this setting, this should result in a mass password reset. I haven't tried this myself though, so not sure. Just a thought.


----------



## DomainBop

> I know from talking with GVH folks overnight, that no one with credentials did anything to generate said emails about passwords.



The GVH children are completely lacking in experience and wouldn't know a security breach if it bit them in the ass and they wouldn't know how to  find the cause of the breach after they were hacked. Their latest  "VP" and top systems admin is a kid fresh out of high school last year with an unimpressive resume that screams "intern material" not "Vice President of Operations"   http://www.linkedin.com/pub/kaushal-subedi/40/554/9b6

I'm going to agree with what Alex LiquidHost said on LET:



> Well, considering that 99% of the providers here are using WHMCS and this thing has not happened to them, I'd say that this is deffinetely not a bug in WHMCS. Either you initiated a mass password reset or you did something, which eventually caused a mass password reset. Or yeah, you've been hacked.



edit:



> I won't comment on that other website, beyond saying some other well known folks were tired of incessant GVH threads


Little unethical kiddie host Jonny Nguyen also pissed off a lot of people on WHT this week (on multiple  threads) when he basically came out and told his shared/reseller clients to go f*ck themselves because they weren't profitable enough for him to care about any more.


 




> I apologize for the inconvenience that you have experienced and I would like to offer an explanation for the issues at hand.
> 
> The reason for why you experienced slower support was because shared/reseller tickets and issues are marked as significantly lower priority to us than our VPS and dedicated hosting tickets as shared/reseller hosting services and plans are no longer something that we offer. We found that this was a better approach than to raise pricing on clients as we've figured that most of our clients would be upset with having to pay more than what they've paid. *We actually do not make any money off of shared/reseller hosting at all* (considering that we have priced our plans SIGNIFICANTLY LOWER than our competitors) and out of the thousands of services that we have, *shared/reseller hosting make up less than 5% of our active products/services*. The only *logical approach* that we could take in order to continue supporting a product in which we don't make any money on and that we no longer offer was to *lower the support priority*. I'm trying to be brutally honest without sugar coating the truth and although we do deeply apologize for the slower support, we had to do what was best for business. We cannot continue to provide priority support service to shared/reseller clients unless we raise our pricing, and unless enough clients contact us asking for prices to be raised in order to increase support priority, raising our shared/reseller prices is not something we are willing to do.
> 
> 
> If you are looking for a mission-critical service, an upgrade to a VPS or dedicated server would be your best bet. We've had very minimal complaints regarding our VPS and dedicated hosting services and the vast majority of our VPS and dedicated clients are happy with the service that they are receiving.


and:





> As I've said before, we're still motivated to provide a good shared/reseller hosting service, however we have to set shared/reseller tickets/issues as lower priority because it's no longer one of our main focuses and also because it makes up less than 5% of our active services. The other 95% is VPS & dedicated services.
> 
> There is only so much you can do and so far you can go for a service you're not making any money from or benefiting from. From a completely honest standpoint, our focus is on our currently active product lines, not shared/reseller hosting. We'll still aim to provide a good shared/reseller service, however there is only so much we can do.
> 
> 
> We've restarting the restoration of accounts to the srv5 shared server and it's still in process of running.





http://www.webhostingtalk.com/showthread.php?t=1368594&highlight=greenvaluehost&page=2

http://www.webhostingtalk.com/showthread.php?t=1365900&highlight=greenvaluehost&page=3

http://www.webhostingtalk.com/showthread.php?t=1368923&highlight=greenvaluehost


----------



## Patrick

rds100 said:


> There is a setting in WHMCS which says how to store the passwords in the database. I imagine if someone accidently changes this setting, this should result in a mass password reset. I haven't tried this myself though, so not sure. Just a thought.


Entirely possible, if you disable md5 passwords or whatever that option is it will reset all users. 



> Disable MD5 Clients Password  This is not recommended as passwords can be decrypted (Disabling this resets all clients passwords)


Maybe they wanted to see all their clients passwords but didn't work well?


----------



## Aldryic C'boas

This stinks of a 'sysadmin' fucking around in a production environment instead of a proper dev area.  The clients should be thankful it wasn't a Solus fat-finger resulting in their VMs being nuked.

Anyone who has paid them with a credit/debit card should be nigh-terrified, though - any WHMCS-based 'provider' that allows automated payments from your card means that said card number is easily readable by anyone that gets access to the WHMCS install.



drmike said:


> Curious about never being a customer.  Perhaps some other reason to be in their database?  Ticketing?  Sale lead?


Interestingly enough, I was bored last summer and did a controlled study on various providers.  Using virgin, aged domains for random (not common-guessable) email addresses, and signed up in various places with to see who might be sharing/selling client information.

Based on what I learned then, let's just say that it's entirely possible GVH received a bunch of client contacts from 'a business partner'.  Email accounts used for one company in particular in that group of bozos seemed to get a rather alarming number of solicitations from the others in the same pocket.


----------



## qps

Aldryic C said:


> any WHMCS-based 'provider' that allows automated payments from your card means that said card number is easily readable by anyone that gets access to the WHMCS install.


Many providers use a gateway with a vault system (for instance, we use Quantum Vault, but there are others) so the credit card number never touches WHMCS.  It uses a token to automatically charge the card in the future if the user chooses to leave it on file for automated charging.


----------



## GelHost

DomainBop said:


> The GVH children are completely lacking in experience and wouldn't know a security breach if it bit them in the ass and they wouldn't know how to  find the cause of the breach after they were hacked. Their latest  "VP" and top systems admin is a kid fresh out of high school last year with an unimpressive resume that screams "intern material" not "Vice President of Operations"   http://www.linkedin.com/pub/kaushal-subedi/40/554/9b6



Reading his linkedin, it seems that he owns and operate another web hosting company.He been advertising in WHT a lot so I don't think he has sold that site yet.

https://neximweb.com/


----------



## GVH-Jon

No it was not a security breach. Client data is completely safe and has not been leaked. We will have a final statement sent out regarding this issue soon.


----------



## texteditor

GVH-Jon said:


> No it was not a security breach. Client data is completely safe and has not been leaked. We will have a final statement sent out regarding this issue soon.


You aren't smart enough to know it isn't a breach, unless you/another GVH-er initiated this yourself, so I'm guessing Ald's assessment is probably spot-on.



Aldryic C said:


> This stinks of a 'sysadmin' fucking around in a production environment instead of a proper dev area.  The clients should be thankful it wasn't a Solus fat-finger resulting in their VMs being nuked.
> 
> Based on what I learned then, let's just say that it's entirely possible GVH received a bunch of client contacts from 'a business partner'.  Email accounts used for one company in particular in that group of bozos seemed to get a rather alarming number of solicitations from the others in the same pocket.


----------



## SkylarM

What's this button do?!


----------



## Tactical

Maybe my "Marine Corps Grunt Brain" still doesn't understand that GVH is saying it was whmcs cronjob that caused the reset? How are you so quick to say it was the automation cronjob in whmcs that reset the passwords? I looked at the cron.php and look encrypted to me with ioncube. Do you have a decrypted version? Did you go through the code to see where it if even makes a call to the database saying reset the passwords?  Im just wondering!


----------



## Aldryic C'boas

qps said:


> Many providers use a gateway with a vault system (for instance, we use Quantum Vault, but there are others) so the credit card number never touches WHMCS.  It uses a token to automatically charge the card in the future if the user chooses to leave it on file for automated charging.


Good to hear there are secure alternatives now (and have possibly been around for awhile - I admittedly have not looked into automated payments, especially with cards, on WHMCS in quite some time.  We don't offer this solely out of distrust of relying on third party software to handle such sensitive information) - though to be quite honest I doubt an amateur operation like GVH would use anything above 'host-in-a-box' panel defaults.



Tactical said:


> GVH is saying it was whmcs cronjob that caused the reset?


That's their official story?  That the automated cron caused the reset?


----------



## texteditor

Tactical said:


> Im just wondering!






Spoiler



They are too


----------



## serverian

Some say love, it is a river.


----------



## Tactical

@aldryic

"said: Just an update, as of now we have no evidence of a "hack" it looks more like a bug in WHMCS as the password resets were sent in line with the Cronjob, we will escalate to WHMCS to see if they have a better insight"

"It appears that WHMCS ran it and its in line with the timing of cronjob finishing another process."  

From LET


----------



## DomainBop

Aldryic C said:


> That's their official story?  That the automated cron caused the reset?


Yep, and they're doing a Fabozzi and blaming it on the software maker.

Quote from their new "Vice President of Operations" :



> Just an update, as of now we have no evidence of a "hack" it looks more like a bug in WHMCS as the password resets were sent in line with the Cronjob, we will escalate to WHMCS to see if they have a better insight


http://lowendtalk.com/discussion/26078/gvh-password-reset/p2


----------



## Aldryic C'boas

Blaming the Cron reeks of bullshit.
 
For starters, the cronfile is ioncube'd, unless they're pulling a Fabozzi and using cracked/unlicensed software.  The cronjob also has a very strict list of items to run, and modifying user accounts is not among those.  The only way the cronjob would've issued password resets, is if some dumbass placed an /includes/hooks/ file to do so.
 
The cron is also only intended to be run once per day.  Which begs an explaination for why Sure, you _could_ set the cron.php file to run multiple times in one day - but then *every* client will receive duplicate emails regarding due services, invoices, and so forth.  And let's not forget that WHMCS logs all actions taken - if this were actually a bug in the cron, the logs would've given evidence of such immediately, and there wouldn't be any uncertainty at all.
 



GVH-Jon said:


> No it was not a security breach. Client data is completely safe and has not been leaked.


That's a pretty bold claim to make when you don't actually know what the problem is.  Unless the prior deduction of 'testing on a production platform' turned out to be true, and now you're just needing a scapegoat to make you look less ridiculous.


----------



## Tactical

I knew it smelled like shit. So what is the real truth? Just say you don't fucking know!


----------



## toadyus

The real truth is they either got hacked or they were fucking around and caused the email to go out. I would be very worried if I was a client and that they have turned off the md5 on all users passwords.

Thank god I never signed up with this horrid provider they have been a joke since day 1!


----------



## Virtovo

SkylarM said:


> What's this button do?!


Does this actually send out password reset emails to all clients?


----------



## SkylarM

Virtovo said:


> Does this actually send out password reset emails to all clients?


I have no idea.


----------



## ChrisM

Soooo. GVH found another way to give themselves free advertising through pass resets? That's unique.


----------



## DomainBop

Virtovo said:


> Does this actually send out password reset emails to all clients?


The WHMCS documentation doesn't say whether it sends out emails but if that option was enabled it poses a huge security risk (mainly because idiots tend to  use the same password everywhere), and GVH clients should seriously consider closing their GVH account because there is no legitimate reason why an admin of an unmanaged service would need to see the passwords of all of their customers

From WHMCS:



> For security client area passwords are irreversibly encrypted and cannot be viewed by admins, enabling this option will switch to reversible encryption allowing admins to view the password. When switching from irreversible to reversible clients will all be assigned a new password and will need to use password recovery.
> http://docs.whmcs.com/Security_Tab#Disable_MD5_Clients_Password


I don't see "password resets"/"change in MD5 ", etc. on the list of cron jobs...


affcommissions
affreports - First day of the month only
backups
cancelrequests
ccexpirynotices
ccprocessing
clientstatussync
closetickets
domainrenewalnotices
emailmarketing
fixedtermterminations
invoicereminders
invoices
latefees
overagesbilling - Runs on the last day of the month only.
suspensions
terminations
updatepricing
updaterates
usagestats
skip_report
http://docs.whmcs.com/Crons


----------



## rds100

I don't know how Mandrill works, but does it add headers that show where (what IP) the email was sent from?

I am wondering whether it was their main WHMCS that sent out the emails, or another one (say their dev WHMCS, where other data from non-customers could have been imported).


----------



## Aldryic C'boas

Did a bit of testing on our WHMCS dev platform..



SkylarM said:


> What's this button do?!


This does reset passwords.  It does not email clients.

Original password, with md5 enabled:


mysql> SELECT password FROM tblclients WHERE id = 1;
+----------------------------------------+
| password |
+----------------------------------------+
| 2ae51bfc8fec24a86a8cbbffc345ccd2:%!Rd% |
+----------------------------------------+
1 row in set (0.00 sec)

Stored password after checking said box, and saving:


mysql> SELECT password FROM tblclients WHERE id = 1;
+------------------------------------------+
| password |
+------------------------------------------+
| I3noCCtmFPCtEFgyxrKVKyPcgctuttmdv4ooxFbV |
+------------------------------------------+
1 row in set (0.01 sec)

Stored password after re-enabling md5, and saving again:


mysql> SELECT password FROM tblclients WHERE id = 1;
+----------------------------------------+
| password |
+----------------------------------------+
| 56570acfefb64055b752e81fb588257f:E#WR# |
+----------------------------------------+
1 row in set (0.00 sec)

It appears WHMCS does randomize new passwords each time this option is toggled (I ran through this loop several times to verify) - however, clients do not get notified of the change.

Here's where things get fun.  There is no documented API function that lets you reset a client's password (which would be necessary for this scenario, since manually updating SQL won't send emails).  There *ARE* API functions for encrypting and decrypting non-md5 passwords, as well as an API function for resetting a _service_ (not WHMCS login) password.

The only method of resetting a client's password that includes an automated, templated email is the `Reset & Send Password` button on the *clientsummary.php* page.  This leaves the following options:


*An actual WHMCS Bug - *I'm inclined to discredit this one.  The actual function to reset the pass and email the client is embedded within the *clientsummary.php* page, as evidenced by the _&resetpw=true&token=_ called on the URL.  The WHMCS cron does *not* call the *clientsummary.php* page at all - and with no other hooks/functions that both reset the pass and issue an email, the chances of this being a cron bug are next to nil.
 
*Poorly Coded Module - *This has some potential.  To reset a client's password, you have to call the *clientsummary.php* page directly with the _&resetpw=true&token=[REDACTED]_ flags included.  So, assuming you were competent with bash/perl/etc, you could fairly easily write a script that would generate a list of URLs to hit with curl that would effectively mass-reset passwords.  I've actually written a script to do just this some time back as a precaution when all of the exploits started surfacing last year.  However - I did say _assuming you were competent_, so I think we can pretty much agree this option is ruled out.
 
*Security Breach - *A frightening possibility - but worth considering.  If someone had malicious thoughts of GVH, but didn't want the innocent clients to become casualties, gaining access and doing something minor like this would be enough to put a scare into folks.

There are a plethora of other scenarios, but most rather unlikely and would be a waste of time to go into.  It'll be pretty easy to see what actually happened as more excuses come forth, though - GVH has a known history of telling blatant exaggerations/bluffs, only to come back later with a different story.


----------



## HH-Josh

I've sent an email to GVH. Will keep you all updated.


- Josh


----------



## rds100

@Aldryic C'boas try running the daily cron after toggling the box, maybe it will send the password reset emails then?


----------



## Aldryic C'boas

rds100 said:


> @Aldryic C'boas try running the daily cron after toggling the box, maybe it will send the password reset emails then?


I tried, it did not.  Sorry, forgot to mention that in the last post.


----------



## SkylarM

Aldryic C said:


> I tried, it did not.  Sorry, forgot to mention that in the last post.


For what it's worth, I still have a dummy account setup with GVH with no VPS active and did NOT receive a password reset email. I did, however, get a mass email that stated they are investigating.


----------



## Aldryic C'boas

SkylarM said:


> For what it's worth, I still have a dummy account setup with GVH with no VPS active and did NOT receive a password reset email. I did, however, get a mass email that stated they are investigating.


Similar situation here.  Which is why I posited the third option, that someone may have just gained access and was randomly resetting client passwords to screw with them.

Now what's going to be absolutely *hilarious*, though - there is another possibility.  There might just be someone using the client-side pass reset function, and GVH didn't properly investigate what was going on before jumping the gun and mass-mailing.  Perhaps it's time that CC starts employing actual professionals instead of kids, and they wouldn't have all of these issues with their child brands.


----------



## toadyus

It would seem that they maybe compromised as people are now getting second and third password reset emails. Or it could be a case of they're just getting them now from previous attempts. This is getting interesting..hopefully their 50+ techs can figure out the issue.


----------



## mtwiscool

i just received 3 emails with in 2 minutes.


----------



## WebSearchingPro

I just got 8 password reset emails starting at 12:01 this afternoon and one randomly every so often. Something is up.


----------



## Aldryic C'boas

WebSearchingPro said:


> I just got 8 password reset emails starting at 12:01 this afternoon and one randomly every so often. Something is up.


Would you mind pastebinning a couple of the emails (or just one if they're all the same), minus the passwords of course?  And possibly the headers from one or two of 'em?  There are actually several different emails from WHMCS relating to password resets, and I'm curious to know exactly which are being sent out.


----------



## mtwiscool

Aldryic C said:


> Would you mind pastebinning a couple of the emails (or just one if they're all the same), minus the passwords of course?  And possibly the headers from one or two of 'em?  There are actually several different emails from WHMCS relating to password resets, and I'm curious to know exactly which are being sent out.


http://pastebin.com/QeNcw9gr


----------



## rds100

Are the passwords in the 8 emails different, or the same? Which one actually works in their billing system - the first one or the last one?


----------



## mtwiscool

rds100 said:


> Are the passwords in the 8 emails different, or the same? Which one actually works in their billing system - the first one or the last one?


Different passwords.


----------



## Aldryic C'boas

mtwiscool said:


> http://pastebin.com/QeNcw9gr


Did the password you were sent work?


----------



## WebSearchingPro

Aldryic C said:


> Did the password you were sent work?


Looks like I got beat the the pastebin, PM'd

The last password sent is valid.


----------



## Aldryic C'boas

Bad news folks.  If GVH's WHMCS lets you change your client details (address, phone, etc) I very strongly suggest doing so now to try and keep your personal info safe.

Given that they likely use WHMCS modules to allow access to the VMs, it would be a wise idea to go set new root passwords as well - and contemplate pulling any sensitive data off of those VPSes.


----------



## rds100

edit: nevermind, brainfart on my side


----------



## couldhave

i have a vps with gvh.  my first password reset was at 1 am.  It did not look random, as the password contained "fu u cnt"  Although it had the appearance of random with numbers letters some capitols etc and is the same length as the automated ones.  Also as others stated i received a slew of automated password resets at about 1 pm eastern.


----------



## toadyus

Aldryic C said:


> *Poorly Coded Module - *This has some potential.  To reset a client's password, you have to call the *clientsummary.php* page directly with the _&resetpw=true&token=[REDACTED]_ flags included.  So, assuming you were competent with bash/perl/etc, you could fairly easily write a script that would generate a list of URLs to hit with curl that would effectively mass-reset passwords.  I've actually written a script to do just this some time back as a precaution when all of the exploits started surfacing last year.  However - I did say _assuming you were competent_, so I think we can pretty much agree this option is ruled out.


Can you run this script as a non-admin tho?


----------



## Aldryic C'boas

toadyus said:


> Can you run this script as a non-admin tho?


With curl, no, as using curl would require actually having an admin login.

BUT.  There are several other ways to accomplish the same thing that would not require a WHMCS admin account.


----------



## KuJoe

Please tell me they at least allow clients to enable Two-Factor Authentication on their WHMCS.


----------



## Aldryic C'boas

KuJoe said:


> Please tell me they at least allow clients to enable Two-Factor Authentication on their WHMCS.


Honestly irrelevant at this point.  I cannot replicate the resets folks are seeing in any other method other than as an admin.  Very good chance they're already compromised beyond the point two-factor would do any good.


----------



## mtwiscool

Aldryic C said:


> Honestly irrelevant at this point.  I cannot replicate the resets folks are seeing in any other method other than as an admin.  Very good chance they're already compromised beyond the point two-factor would do any good.


how do you know if its been compromised?


----------



## Aldryic C'boas

mtwiscool said:


> how do you know if its been compromised?


Because if this were just a simple case of someone abusing the client-side password reset (like these guys seem to think), you would be receiving two emails.  The first being a confirmation email that would have you click a URL containing a randomized token - doing so would perform the actual password reset, which you'd receive in the second email.

When an admin resets your password, all you receive is the second email (the one you pastebin'd for me).

Someone charitable might want to cross post over to LE and let those folks know that it's a bit more serious than they're assuming.


----------



## MartinD

Please stick to the topic at hand. Thanks.


----------



## raindog308

Aldryic C said:


> This stinks of a 'sysadmin' fucking around in a production environment instead of a proper dev area.


Hmmm...I think having a dev instance is only allowed with a purchased WHMCS license (as opposed to leased).  I guess I wouldn't be surprised if GVH leases WHMCS.


----------



## Aldryic C'boas

raindog308 said:


> Hmmm...I think having a dev instance is only allowed with a purchased WHMCS license (as opposed to leased).  I guess I wouldn't be surprised if GVH leases WHMCS.


Not unless that's a newish rule?  I've had a dev license for years, and we've only owned our WHMCS license for the past.. maybe 15-20 months?

EDIT:  Pretty sure the catch is they'll only provide support if you keep your owned license renewed - leased licenses are pretty much good to go.


----------



## couldhave

@aldryic   I had done my own password resets at gvh earlier today and it did not require a second step or clicking a link. To change the password only required you to input an email address and click the reset password link.  You would than get an email containing you your new password.   However I just notice something very interesting.  At the end of the slew of password resets was the final one which DID require clicking a link.  So it looks like they were updating things on their end.  The only thing that worries my is my initial password having "fu u cnt" in it.  Maybe the guys at gvh thing this is funny, or maybe there was some compromise earlier.


----------



## raindog308

I wonder if someone:

 

1. took a list of emails from some provider's previous leak (e.g., CVPS database)

2. wrote a script to request a password reset for each email from GVH

3. let it run endlessly

 

In which case it'd be easy for a competent admin to identify where all the resets are coming from and block...


----------



## couldhave

i didn't look at the emails close enough.....

1:02 am password in email

moving to PM....

1:02 pm password in email

1:04 requires a link

1:05 password in email

1:06 password in email

1:06 password in email

1:07 password in email

1:08 password in email

1:09 password in email

1:11 requires a link

During this time i am almost certain I requested two passwords resets myself, which would probably be the 1:04 and 1:11.


----------



## Aldryic C'boas

couldhave said:


> @aldryic   I had done my own password resets at gvh earlier today and it did not require a second step or clicking a link. To change the password only required you to input an email address and click the reset password link.  You would than get an email containing you your new password.   However I just notice something very interesting.  At the end of the slew of password resets was the final one which DID require clicking a link.  So it looks like they were updating things on their end.  The only thing that worries my is my initial password having "fu u cnt" in it.  Maybe the guys at gvh thing this is funny, or maybe there was some compromise earlier.


That's... rather frightening when you stop to think about it.  WHMCS has included password verification (the two-email process) for quite some time - and unless I'm blind, there's not an option to disable.  Which makes me wonder just how *old* their WHMCS install was.



raindog308 said:


> I wonder if someone:
> 
> 
> 
> 1. took a list of emails from some provider's previous leak (e.g., CVPS database)
> 
> 2. wrote a script to request a password reset for each email from GVH
> 
> 3. let it run endlessly
> 
> 
> 
> In which case it'd be easy for a competent admin to identify where all the resets are coming from and block...


That was my first thought.. but I haven't found an option to disable the verification emails for password resets.  Meanwhile, an admin issuing the reset by hand automatically generates the new pass and sends the email.


----------



## Aldryic C'boas

couldhave said:


> i didn't look at the emails close enough.....
> 
> 1:02 am password in email
> 
> moving to PM....
> 
> 1:02 pm password in email
> 
> 1:04 requires a link
> 
> 1:05 password in email
> 
> 1:06 password in email
> 
> 1:06 password in email
> 
> 1:07 password in email
> 
> 1:08 password in email
> 
> 1:09 password in email
> 
> 1:11 requires a link
> 
> During this time i am almost certain I requested two passwords resets myself, which would probably be the 1:04 and 1:11.


Aah, that makes more sense.  But, still means that someone gained access to an admin account, and was having a gaye old time with it.

What's truly disturbing is how long it's gone on without them having the sense to track down the issue, or even block public access to prevent further damage.


----------



## rds100

I don't know how many clients they have, but i can't imagine someone (an admin) being able to go through the list of all their clients and manually click on the "reset & set password" link for each client, all this in a minute or so. Must have been scripted / automated.


----------



## WebSearchingPro

Aldryic C said:


> What's truly disturbing is how long it's gone on without them having the sense to track down the issue, or even block public access to prevent further damage.


I just talked to Jon, their sysadmin blocked off whmcs admin to prevent anymore resets. I personally was tired of getting spammed with password emails .


----------



## Aldryic C'boas

rds100 said:


> I don't know how many clients they have, but i can't imagine someone (an admin) being able to go through the list of all their clients and manually click on the "reset & set password" link for each client, all this in a minute or so. Must have been scripted / automated.


Depends on how many clients they actually have (that group is known to exaggerate figures), and whether or not it was actually a 'mass' reset as opposed to just spamming a bunch of random clients' reset links.


----------



## mtwiscool

WebSearchingPro said:


> I just talked to Jon, their sysadmin blocked off whmcs admin to prevent anymore resets. I personally was tired of getting spammed with password emails .


they whmc is still accessible.


----------



## Virtovo

mtwiscool said:


> they whmc is still accessible.


Their WHMCS is, their admin path is not.


----------



## Aldryic C'boas

Virtovo said:


> Their WHMCS is, their admin path is not.


Which pretty much confirms that they know it was someone running around with a compromised admin account.  If they truly thought it was a bug in WHMCS, they would either throw maintenance mode, or lock the entire site down.


----------



## DomainBop

> What's truly disturbing is how long it's gone on without them having the sense to track down the issue



They need to hire someone competent to deal with the issue.  Jon doesn't have the necessary skills to deal with a security breach and I don't have much confidence in his new 'VP of Ops' (granted my judgement of the new VP might be clouded by the "DCMA" _[sic]_ graphic on his site)


----------



## Aldryic C'boas

If you have to give someone a title like 'VP', you already know they're worthless for real work.


----------



## WebSearchingPro

Aldryic C said:


> If you have to give someone a title like 'VP', you already know they're worthless for real work.





> I don't have much confidence in his new 'VP of Ops'



Their VP of Ops is a very talented admin, however who it is is confidential unfortunately. Just know you are in good hands!


----------



## Aldryic C'boas

"We're letting someone new poke around our systems and view all of your data.  We can't tell you who it is, though."


----------



## Virtovo

Aldryic C said:


> "We're letting someone new poke around our systems and view all of your data.  We can't tell you who it is, though."


Probably the same guy who was sending password resets.  Admin path is on WHMCS default also.


----------



## MartinD

WebSearchingPro said:


> Their VP of Ops is a very talented admin, however who it is is confidential unfortunately. Just know you are in good hands!


See if you're going to throw cloak and dagger crap around, don't bother posting. Either give the information out or say nothing. All you're doing is making yourself look incompetent because no decent admin would put that kind of pish on their own website.


----------



## DomainBop

WebSearchingPro said:


> Their VP of Ops is a very talented admin, however who it is is confidential unfortunately. Just know you are in good hands!


It can't be too confidential since "VP of Operations GreenValueHost" is posted on his LinkedIn profile for everyone to see (see the link I posted on the first page of this thread ).


----------



## WebSearchingPro

MartinD said:


> See if you're going to throw cloak and dagger crap around, don't bother posting. Either give the information out or say nothing. All you're doing is making yourself look incompetent because no decent admin would put that kind of pish on their own website.


It was just more of a reassuring statement, I apologize, they are an active member in this community and I would not want to risk their anonymity. I'd rather not be hassled by them when they see their name . The information is freely out there, just have to dig it up.


----------



## MannDude

Wow, I just woke up. Read all of this thread. Read all of the LET thread. How is no one there concerned?

"La la la. I keep getting these emails, LOL!! La la la! Doo do doo"

I'd say it's quite worth being worried about.


----------



## Tactical

Is school out yet?


----------



## AuroraZero

MannDude said:


> Wow, I just woke up. Read all of this thread. Read all of the LET thread. How is no one there concerned?
> 
> "La la la. I keep getting these emails, LOL!! La la la! Doo do doo"
> 
> I'd say it's quite worth being worried about.


They are not concerned because they know GVH will just throw some free services at them for the inconvience. Everyone loves free stuff. They have not realized that this is the wake up call I have seen coming for months. Not just for them but for a lot of other companies as well if they do not change their setups. This is not a game no matter how many people think it is at this time.

Not trying to be an ass or anything but it is the truth.


----------



## MannDude

Comically there is no WHT thread regarding this. Was it only LET customers who got their passwords reset?..

Someone should post on WHT too and let it be known that GVH customers probably need to secure their data.


----------



## kaniini

GVH-Jon said:


> No it was not a security breach. Client data is completely safe and has not been leaked. We will have a final statement sent out regarding this issue soon.


What was it then?  I'd be dying to hear your analysis.


----------



## hellogoodbye

I'm a former client (canceled sometime in early March) and I didn't receive any emails about my password being reset, just the one email that was sent out informing people they were investigating the issue. Is this only happening to people with active services?


----------



## Aldryic C'boas

hellogoodbye said:


> I'm a former client (canceled sometime in early March) and I didn't receive any emails about my password being reset, just the one email that was sent out informing people they were investigating the issue. Is this only happening to people with active services?


It seems that the actual resets are happening to clients at random - which pretty much just reinforces the idea that it was someone running wild with a compromised staff account.


----------



## MannDude

hellogoodbye said:


> Is this only happening to people with active services?


Apparently not. Someone else mentioned it had happened to them, and they were not a customer at all.

Also the lack of chatter on WHT would indicate it wasn't system wide, but instead isolated to groups.


----------



## hellogoodbye

That's interesting... the timing is quite odd too, I remember at some point last night there was an onslaught of old GVH threads being dug up over at LET by a few members who I guess was taking the "LET = GVH" joke too far. I don't see most of those threads anymore so I'm assuming the admins stepped in and cleaned it up/doled out punishment where necessary. This might just all be coincidence, but could there possibly be a connection between the two?


----------



## serverian

hellogoodbye said:


> That's interesting... the timing is quite odd too, I remember at some point last night there was an onslaught of old GVH threads being dug up over at LET by a few members who I guess was taking the "LET = GVH" joke too far. I don't see most of those threads anymore so I'm assuming the admins stepped in and cleaned it up/doled out punishment where necessary. This might just all be coincidence, but could there possibly be a connection between the two?


Yeah, me and Spencer did it!


----------



## hellogoodbye

serverian said:


> Yeah, me and Spencer did it!


lol.

But in all seriousness, I'd suggest being more careful with your words; people might actually take you seriously.


----------



## Virtovo

hellogoodbye said:


> lol.
> 
> But in all seriousness, I'd suggest being more careful with your words; people might actually take you seriously.


No really they did.  Dragging up threads that is, not sending password resets


----------



## toadyus

serverian said:


> Yeah, me and Spencer did it!


I lol'd so hard last night!!


----------



## hellogoodbye

Virtovo said:


> No really they did.  Dragging up threads that is, not sending password resets


Ahh gotcha. In that case I wasn't too sure who started it, though my guess was @serverian since I noticed the banned user icon when I was skimming through the updated threads just now.


----------



## rds100

serverian said:


> Yeah, me and Spencer did it!


So you were working undercover for that outsourcing support firm in India, that's how you got access to an admin account for their WHMCS?


----------



## raindog308

I'm horrified/amused that this site exists: http://www.gvhtalk.com 

"Now that GreenValueHost has finally reached the point of stability inside and out..."


----------



## serverian

rds100 said:


> So you were working undercover for that outsourcing support firm in India, that's how you got access to an admin account for their WHMCS?


All was a part of a masterplan! TO RESET THOSE DAMN PASSWORDS


----------



## DomainBop

serverian said:


> Yeah, me and Spencer did it!


Don't try to blame Spencer.  I read in the cestpit it was ALL YOUR FAULT! 



> After having read through both IRC logs for #lowendbox and #vpsboard, I've decided to lift Spencer's ban and just give him a warning.
> 
> serverian was the "organizer" behind all this. His ban remains, he is also not allowed to bla bla bla...


----------



## MannDude

Let's keep this on topic if possible and discuss the likely GVH security breach. Thanks.


----------



## serverian

DomainBop said:


> Don't try to blame Spencer.  I read in the cestpit it was ALL YOUR FAULT!


Yeah, I'm the root of the evil! Poor Spencer was just a minion.


----------



## hellogoodbye

Sorry about that!

Either way, I'm looking forward to hearing an official explanation on the matter, especially after they initially placed the blame on an out-of-control cronjob. Since the comment about them making an official statement "soon" was made over six hours ago, I hope it won't be too long now.


----------



## DomainBop

MannDude said:


> Let's keep this on topic if possible and discuss the likely GVH security breach. Thanks.


What data breach? We're here to discuss WHMCS's bad coding and the out of control cron job!.

If there had been a data breach (which there wasn't because Jon said so),  then Jon would need to notify all of his customers and follow the notification procedures that 46 states have in place.   If there had been a breach (which there wasn't) then Jon would be wise to read the information on these 2 links: http://www.perkinscoie.com/statebreachchart/ and http://usa.visa.com/download/merchants/cisp_responding_to_a_data_breach.pdf


----------



## jarland

Dear lord...

https://copy.com/ulZUpFl1nzxpM3U3


That's only about half of them too. If their host had a certain mascot in Florida I'd tell them what happened in 3 minutes


----------



## drmike

So delving slightly off topic here...

There was a WHMCS link posted to IRC earlier.  Rogue-Oktay posted and asked people to ping it... Some did/looked and bunch of background crap happened. Growl.

cron.php

To those of you familiar with such / with installs... Can you access this file from remote locations?   Require credentials in generic install?

WTF is that file normally?


----------



## serverian

drmike said:


> So delving slightly off topic here...
> 
> There was a WHMCS link posted to IRC earlier.  Rogue-Oktay posted and asked people to ping it... Some did/looked and bunch of background crap happened. Growl.
> 
> cron.php
> 
> To those of you familiar with such / with installs... Can you access this file from remote locations?   Require credentials in generic install?
> 
> WTF is that file normally?


Nah, I thought that was the thing that triggered the mails and posted the link. It seemed the file was locked out of public access already.


----------



## alexvolk

drmike said:


> So delving slightly off topic here...
> 
> There was a WHMCS link posted to IRC earlier.  Rogue-Oktay posted and asked people to ping it... Some did/looked and bunch of background crap happened. Growl.
> 
> cron.php
> 
> To those of you familiar with such / with installs... Can you access this file from remote locations?   Require credentials in generic install?
> 
> WTF is that file normally?


http://pastebin.com/tjkjws2q

Later will have a better code  

Edit: this is from whmcs 5.2.10 decoded but still code is similar to latest one ones like 5.3.*: https://raw.githubusercontent.com/kiddo90/whmcs_5.2.10_decoded_nulled_mtimer/master/admin/cron.php

Edit 2: after analyzing the cron.php code I would like to say that there is nothing what could make a password reset at all.

If somebody had an admin account it's possible to reset password with api like this:

1) Update client password http://docs.whmcs.com/API:Update_Client

2) Send email with custom email template (it does support custom variables and etc) http://docs.whmcs.com/API:Send_Email#Example_with_Predefined_Email_Template


----------



## serverian

I don't see anything that would trigger password resets on that file ^ But I maybe blind.


----------



## Aldryic C'boas

drmike said:


> So delving slightly off topic here...
> 
> There was a WHMCS link posted to IRC earlier.  Rogue-Oktay posted and asked people to ping it... Some did/looked and bunch of background crap happened. Growl.
> 
> cron.php
> 
> To those of you familiar with such / with installs... Can you access this file from remote locations?   Require credentials in generic install?
> 
> WTF is that file normally?


Cron.php is an ioncube'd file that contains WHMCS 'daily chores'.  Calculating totals, invoice generation, email notifications (for invoices, etc) - pretty much most of your automated tasks.

It can be accessed 'remotely' if the admin is less-than-competent, and doesn't secure his install.  However, the cron.php file itself does not take any arguements or POST data - it's not really susceptible to injection or true malicious use by itself.  But, the file is intended to be accessed once a day.  If it's publicly accessible, and it gets spammed, and someone has a due invoice - that someone will receive a new 'invoice overdue' email each time it gets hit.

By itself, really the worst you can do is troll a company that doesn't know how to properly secure WHMCS.  I elaborated earlier on why blaming the cron for the password resets was utter BS.


----------



## drmike

Thank you much @alexvolk!

I see nothing in there about passwords or any relationship.

Normally folks bury this stuff behind HTACCESS or firewall rules or similar, right?

Thanks Ald.


----------



## drmike

I can confirm some of what happened and purely from WHMCS logging.

2 IPs at Amazon poked something somewhere in WHMCS.  This fired off a bunch of email/password activity.  Picture is worth a 1000 words, so screencap time:


----------



## drmike

Other IP doing similar activity:


----------



## drmike

You aren't going to see this over on that Lowendshit site 

1 more coming.


----------



## alexvolk

drmike said:


> I can confirm some of what happened and purely from WHMCS logging.
> 
> 2 IPs at Amazon poked something somewhere in WHMCS.  This fired off a bunch of email/password activity.  Picture is worth a 1000 words, so screencap time:



So it's definitely related to Cron - > Hooks.



Code:


$cron->logActivity("Completed");
$cron->emailReport();
run_hook("DailyCronJob", array());
$cron->log("Cron Job Hooks Run...");
if ($cron->isScheduled("backups")) {


----------



## hellogoodbye

If it's this obvious that a couple of suspicious IPs were behind the WHMCS tinkering, why did they initially explain it away as a WHMCS bug?


----------



## Amitz

Where are those logfiles from? Damn, what did I miss?


----------



## drmike

hellogoodbye said:


> If it's this obvious that a couple of suspicious IPs were behind the WHMCS tinkering, why did they initially explain it away as a WHMCS bug?


WHMCS was contacted this morning and basically hasn't a clue.  Unsure what we are collectively looking at.  My guess would be injected WHMCS install - as in something injected rogue. Script or config...  But that's 100% speculation on my part.



Amitz said:


> Where are those logfiles from? Damn, what did I miss?


I got those captures directly from GVH administrators.


----------



## drmike

So...

The custom hooks theory:

*" its not hook related... [GVH] doesn't use any custom hooks"*


----------



## jarland

How does one not log all web server activity, or if one does, how does one not instantly know what happens?


----------



## alexvolk

drmike said:


> So...
> 
> The custom hooks theory:
> 
> *" its not hook related... [GVH] doesn't use any custom hooks"*


If they didn't use any hooks and checked the logs that no files has been uploaded - then gvh is safe (not hacked) I think.

What comes to mind is to submit password reset like a real user (automated) and at the same visit cron url - the reason is to simply hide that client database of one of providers was used.

Http logs will make it clear.

I would like to say that on older version of whmcs there were simply password reset on request, no verification for the link is needed. If they were updating their whmcs step by step (version by version) then there is a big chance like 99% that it was still the same even having latest whmcs.


----------



## drmike

Well, like all web stuff, logging is on...  That would be within WHMCS as well as underlying web server logs.

I try to always be straight with info and honest with folks as to what they are seeing. No real time to vet this info to death like normally.  So things to note, live time:

1. The screencaps above were from NOON not midnight.

2. NOON correlates to when IRC posting happened - not claiming it has any relationship big picture. Suspect just A+B + bad timing.

3. Original emails started shooting out at midnight, by others accounts overnight.

4. There were more emails after NOON event.


----------



## hellogoodbye

What did the logs look like from the original midnight event? Were the same IPs behind the emails, or did it show different IPs?


----------



## drmike

Another screencap, with activity on one of the IPs - this is from the web server logs:


----------



## jarland

drmike said:


> Another screencap, with activity on one of the IPs - this is from the web server logs:


Odd no POST in there, unless it's elsewhere.


----------



## drmike

Note the runscope part...

https://www.runscope.com/

"Testing on your terms.

Create tests based on actual calls made from within your apps. Capture traffic from any app without changing your code. Define test criteria with flexible assertions and variables to pass state between requests."

API debugging outsourced service.

Definitely sounds / seems like hacking at URL construction.


----------



## drmike

And note @jarland, they were 200 OK then went 404... That was when rules were put into place and firewall erected (i.e. directory blocked externally).


----------



## HH-Josh

Issue resolved as regards to my account. Appears that someone in the office signed up using my details for a service we never used, hence why my email is listed with GVH - as stated before I've never personally been a client so seemed a bit strange as to why I was listed, more to do with a staff member in the office not communicating with me.


GVH now seem to think I have a vendetta against them to try and ruin their reputation, which is completely false and un true. I never have, or will be a client of GVH, so it would be completely unfair to judge them in any way.


Hope to hear more of an update shortly though as regards to what's been going on - quite an interesting situation.


- Josh


----------



## raindog308

I'm thinking that perhaps I want to add a CSF rule for WHMCS...too many password reset requests from an IP in X time is a temp block.


----------



## Nett

GVH-Jon said:


> No it was not a security breach. Client data is completely safe and has not been leaked. We will have a final statement sent out regarding this issue soon.


How could you tell?


----------



## DomainBop

drmike said:


> And note @jarland, they were 200 OK then went 404... That was when rules were put into place and firewall erected (i.e. directory blocked externally).








It's a little late to be putting firewall rules in today when they've been in operation for 2 years. They should have secured that directory when they installed WHMCS (not to mention changing the admin folder name).  The WHMCS documentation also suggests moving the crons folder to a non publically accessible folder, password protecting the admin directory, limiting access by IP, etc.  http://docs.whmcs.com/Further_Security_Steps#Change_your_WHMCS_Admin_Folder_Name

The screenshot basically confirm what we all know: The kids at GVH aren't the brightest bulbs on the security block, and failed to properly secure their site and as a result they put their users info at risk .  I would ask if they do regular PCI/vulnerability scans of their WHMCS and SolusVM installations but I think I already know the answer to that one: NO.


----------



## jarland

DomainBop said:


> It's a little late to be putting firewall rules in today when they've been in operation for 2 years. They should have secured that directory when they installed WHMCS (not to mention changing the admin folder name). The WHMCS documentation also suggests moving the crons folder to a non publically accessible folder, password protecting the admin directory, limiting access by IP, etc. http://docs.whmcs.com/Further_Security_Steps#Change_your_WHMCS_Admin_Folder_Name
> 
> 
> The screenshot basically confirm what we all know: The kids at GVH aren't the brightest bulbs on the security block, and failed to properly secure their site and as a result they put their users info at risk . I would ask if they do regular PCI/vulnerability scans of their WHMCS and SolusVM installations but I think I already know the answer to that one: NO.


Meh, you know PCI compliance and most vulnerability scans are useless and serve mostly to milk money from those who don't know any better. In this market it's both easier and more beneficial to just not be stupid and read documentation...which so many people just can't seem to be bothered to do.


----------



## drmike

Best practices are swell.  But shit happens for stupidity sake and for plain ole never read the farking manual.

I have no defense of the actions or lack thereof.  Seen bad outcomes from compliant and non compliant folks.

Even if wide open and cron.php right out front, should require credentials and other stuff.  There is some disconnect or gap between what happened and what we all think about how WHMCS works.   Wish I had the answer on this one.

A ticket exists at WHMCS for the matter.

I believe in March release, WHMCS had update(s) to the cron.php file.

GVH is running licensed version and current.

So... ahh yeah... if this is real issue, everyone wide open could be smacked in same way.  Folks ought to get to securing installs and generally other communities should be made aware of this issue.


----------



## SkylarM

drmike said:


> So... ahh yeah... if this is real issue, everyone wide open could be smacked in same way.


While there is SOME fault to WHMCS for it being so easy to run crons/pop importer email files, etc the real blame lies in anyone dumb enough to not secure their admin directory. WHMCS even suggests some basic securing of the admin directory: http://docs.whmcs.com/Further_Security_Steps


----------



## jarland

It's really incredible what you can do with a little apache and some .htaccess. If you're willing, you can effectively mod_rewrite yourself out of most common uses for mod_sec, and I wish more people understood apache doesn't execute a php cron in your crontab, it doesn't need to be accessible via apache, or whatever other hipster thing you've got listening on 80.


----------



## DomainBop

> Meh, you know PCI compliance and most vulnerability scans are useless and serve mostly to milk money from those who don't know any better


They're not as effective as practicing basic security on a daily basis but quarterly scans are a requirement for (Internet) merchant accounts [yes, I realize most low end providers don't have merchant accounts].  Sometimes they might even discover a vulnerability that the site owner missed. As far as cost, the daily scans are a money cow for the scan providers but every merchant account provider I've used has thrown in the quarterly scans for free so it is possible to get scanned semi-regularly without spending anything . Comodo's hackerguardian also has a "5 free scans over 3 IPs" free trial so anyone could have their site scanned for vulnerabilities before opening to the public.



> just not be stupid and read documentation...which so many people just can't seem to be bothered to do.


looks at GVH and I'm sure many more "push button" script users (in all online industries)..



> the real blame lies in anyone dumb enough to not secure their admin directory.


The basic "further security steps" suggested by WHMCS would take at most 5 minutes and are so basic that even the most technically inept person could do them so there is really no excuse for not doing them.


----------



## jarland

Well I gotta say props to GVH for sharing screenshots of whmcs log and access logs. That's so much more than I'd expect of some others. Maybe this issue is more complex than it looks. No excuses for some security failures but if I asked everyone who has made a mistake or oversight to raise their hand, I would have mine up as well.


----------



## GVH-Jon

We have identified the root cause and will be releasing a public explanation statement shortly.


----------



## drmike

This is precursor to their coming out party for this issue.

There was no hack.   Nothing was compromised.  Nothing more than cron.php being public and when accessed set off a job in there.

Call it bad configuration + a queued marketing email campaign with a bad toggle option.

Fire of request to cron.php and presto, emails go out in mass... Every time.


----------



## Kris

drmike said:


> Fire of request to cron.php and presto, emails go out in mass... Every time.



Are you sure? Have you been able to duplicate this, or speculation? 

The logs you posted was from mid-day, when GET (not post) was hitting cron.php. Maybe an angry ex-employee with API details.

In terms of the scanner hitting them 12 hours after the emails, seems like a nice way to build evidence for their story that it was cron... by scanning your own site. 

Things have seemed a bit off since you 'helped and were chatting with HVH' and the CC goons, now you're the go-to for GVH help, defending. 

I have to ask... How much was your price?  

*EDIT: From the explanation below, seems feasible actually. Good on GVH for a fast investigation. *


----------



## GreenHostBox

Got this in my email below. I guess this is solved now.



> As you are aware, GreenValueHost experienced issues in the past 24 hours where multiple password resets were sent to customers. We apologize for the flurry of emails.
> 
> 
> 
> Code:
> 
> 
> GreenValuehost was NOT hacked, there wasn't a security compromise either.  Customers are safe and secure.
> 
> What happened involves the WHMCS billing panel.
> 
> Two files: cron.php and /admin were accessible to the public.  These should have been secured with additional rules.  Yes, we have since added multiple layers of security to protect these files from public access.
> 
> A marketing email was pre-wrote and placed in WHMCS for a 2014 Spring Overstock Sale.  The default template was erroneously set to / left at “Automated Password Reset”.  WHMCS defaults the default template pull down option to “Automated Password Reset”.
> 
> Marketing emails run from cron.php events.   Therefore, when someone accessed the cron.php file it triggered the sending of the marketing email, which was then set as “Automated Password Reset”.  This happened a total of 17 times, generating 17 emails per email account.
> 
> GreenValueHost debugged this issue by manually running the secured cron.php, analyzing a few emails sent and looking in WHMCS.
> 
> We have a ticket with WHMCS which will be appended to reflect the debugging and resolution with recommendations to prevent this in future with other WHMCS users (beyond simply securing said files).
> 
> We welcome any customers concerned about the matter or who may be experiencing password problems to submit a ticket.
> 
> 
> Thank You
> GreenValueHost Team


----------



## SkylarM

Not sure why they are using Email Marketer for this, that's not really the intent of the marketer deal. Idea is to say (for example) force password resets every 90 days, or to send an email thanking a customer when they reach X years hosted with you. Not to send out an email that everyone is going to get anyways. At least that's all it was.


----------



## DomainBop

> There was no hack.   Nothing was compromised.



Depends on your definition of "hack"  If you go by the common street meaning "information theft" then there was no known hack (that has been discovered or acknowledged) but if you go by 18 U.S.C. § 1030 then  it would probably qualify as a hack.



> 18 U.S.C. § 1030(a)(5) : Damaging a protected computer
> (5)
> 
> 
> (A) knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer;
> 
> 
> ( B  intentionally accesses a protected computer without authorization, and as a result of such conduct, recklessly causes damage; or
> 
> 
> © intentionally accesses a protected computer without authorization, and as a result of such conduct, causes damage and loss.



("protected computer" is basically any computer/server used in interstate commerce: e.g. a hosting site, ecommerce store, etc)

Wacking the cron repeatedly, and the password reset emails would qualify as intentionally accessing the computer to cause damage.

Regardless of which definition of hacking you use GVH is still at fault for not taking steps to properly secure their server and their negligence does put their users at risk.  I would seriously suggest that they hire a security firm (like Rack911, etc) to thoroughly audit their servers and do some  basic security hardening.

edited to add: @MannDude WTF is up with the formatting on this board


----------



## drmike




----------



## Nett

Hello Name,


As you are aware, GreenValueHost experienced issues in the past 24 hours where multiple password resets were sent to customers. We apologize for the flurry of emails.

_*GreenValuehost was NOT hacked, there wasn't a security compromise either. Customers are safe and secure.*_

What happened involves the WHMCS billing panel.

Two files: cron.php and /admin were accessible to the public. These should have been secured with additional rules. Yes, we have since added multiple layers of security to protect these files from public access.

A marketing email was pre-wrote and placed in WHMCS for a 2014 Spring Overstock Sale. The default template was erroneously set to / left at “Automated Password Reset”. WHMCS defaults the default template pull down option to “Automated Password Reset”.

Marketing emails run from cron.php events. Therefore, when someone accessed the cron.php file it triggered the sending of the marketing email, which was then set as “Automated Password Reset”. This happened a total of 17 times, generating 17 emails per email account.

GreenValueHost debugged this issue by manually running the secured cron.php, analyzing a few emails sent and looking in WHMCS.

We have a ticket with WHMCS which will be appended to reflect the debugging and resolution with recommendations to prevent this in future with other WHMCS users (beyond simply securing said files).

We welcome any customers concerned about the matter or who may be experiencing password problems to submit a ticket.


Thank You
GreenValueHost Team


----------



## roykem

So it's secured now. 

well, can't imagine if after this we keep receiving another pass reset emails from GVH.

hope not.

hey.. i'm new here


----------



## DomainBop

_*"there wasn't a security compromise either.*_" and then in the next paragraph *"Two files: cron.php and /admin were accessible to the public. These should have been secured with additional rules"* 

umm, if someone was able to access the files because they weren't properly secured that's a security compromise


----------



## drmike

DomainBop said:


> _*"there wasn't a security compromise either.*_" and then in the next paragraph *"Two files: cron.php and /admin were accessible to the public. These should have been secured with additional rules"*
> 
> umm, if someone was able to access the files because they weren't properly secured that's a security compromise


Think of it this way, there is a door as a barrier, they just added  a deadbolt lock.  Prior the door opened freely, but was a door. 

Normally shouldn't have been any harm in curling said URL's/files... but in this instance, yeah did what it did, every time.

Nothing was disclosed to anyone other than account holders, in their inbox, their account name + new password.

Certainly an annoyance, PITA and perhaps access issues for clients. 

I'll jump in any time anyone thinks Solus or WHMCS or Cpanel are compromise culrpit and is about to set off mass provider paranoia, pulling of panels, etc.   Saw what happened elsewhere with Solus and the shitfest, additional workload on folks and general distrust in segment. If I can be of some use, I try.


----------



## serverian

drmike said:


> Think of it this way, there is a door as a barrier, they just added  a deadbolt lock.  Prior the door opened freely, but was a door.
> 
> Normally shouldn't have been any harm in curling said URL's/files... but in this instance, yeah did what it did, every time.
> 
> Nothing was disclosed to anyone other than account holders, in their inbox, their account name + new password.
> 
> Certainly an annoyance, PITA and perhaps access issues for clients.
> 
> I'll jump in any time anyone thinks Solus or WHMCS or Cpanel are compromise culrpit and is about to set off mass provider paranoia, pulling of panels, etc.   Saw what happened elsewhere with Solus and the shitfest, additional workload on folks and general distrust in segment. If I can be of some use, I try.


No. When you run the cronjob, it shows you information about overdue invoices like

XX Overdue Invoice Reminders Sent

- Sent First Notice to User Firstname Lastname

- Sent First Notice to User Firstname Lastname

- Sent First Notice to User Firstname Lastname

 

So the ones who ran the cron.php saw some of their customers' names.


----------



## drmike

serverian said:


> No. When you run the cronjob, it shows you information about overdue invoices like
> 
> XX Overdue Invoice Reminders Sent
> 
> - Sent First Notice to User Firstname Lastname
> 
> - Sent First Notice to User Firstname Lastname
> 
> - Sent First Notice to User Firstname Lastname
> 
> 
> 
> So the ones who ran the cron.php saw some of their customers' names.


Ah, what client names did you see?   Someone have a capture?

Interesting.  I won't discount this as that's how the job like runs and you know better than I.  Assumes someone ran from terminal and all though and got output.   There were requests / encouraged people to run the URL.  All is possible.

From bits seen, was an API testing site used to make most requests.  Can't say what they got there, but yeah, possible.


----------



## Mun

LOL I got two emails notifying me it wasn't a hack..


----------



## drmike

Mun said:


> LOL I got two emails notifying me it wasn't a hack..


Really happened?   I'll ask


----------



## Mun

Yep one went to spam, and the other went to my inbox XD


----------



## drmike

Mun said:


> LOL I got two emails notifying me it wasn't a hack..


Mun, you have multiple accounts with multiple emails there.  That's the double email origin for you.


----------



## Mun

drmike said:


> Mun, you have multiple accounts with multiple emails there.  That's the double email origin for you.


Yep indeed, which is equally scary that they let you know this.... Huh, on the bright side my main account now thinks GVH is spam XD


----------



## serverian

drmike said:


> Mun, you have multiple accounts with multiple emails there.  That's the double email origin for you.


You are working for GVH now?


----------



## drmike

Mun said:


> Yep indeed, which is equally scary that they let you know this.... Huh, on the bright side my main account now thinks GVH is spam XD


Unsure what they divulged there that you didn't   Literally was just what I said. 

I ask: "Mun on vpsB got multiple emails from email broadcast, WHY?"

Response: "[he] has multiple emails in system"

Inferred on my part: multiple accounts.

Mind you, people want results and not more misinfo / fear / phobias.

Been a frightful day for providers following along.  Any twitch and someone needs to be on top of things.


----------



## drmike

serverian said:


> You are working for GVH now?


Nope and never have.  That includes likewise working for, consulting, etc. for associated upstreams, or "related" other companies.

$0 taken from, billed to, paid from... No freebies, no complimentary services, nothing. Zippo daddy.


----------



## peterw

drmike said:


> Mun, you have multiple accounts with multiple emails there.  That's the double email origin for you.


They told you information about a customer?


----------



## Hxxx

peterw said:


> They told you information about a customer?


c'mon, we all know drmike is privileged everywhere... He is like the Wikipedia guy


----------



## GelHost

Chris Miller said:


> Soooo. GVH found another way to give themselves free advertising through pass resets? That's unique.


Their marketing team is just too brilliant. Getting their name out any way possible, just brilliant.


----------



## Aldryic C'boas

I wouldn't call advertising your incompetence and poor security to be 'brilliant'.  Not to say I'm unappreciative of their efforts, however - we had a number of new signups last night/today from people no longer trusting teenagers, and wanting a reliable host.


----------



## DomainBop

GelHost said:


> Their marketing team is just too brilliant. Getting their name out any way possible, just brilliant.


Vitaly Borker is probably the most famous proponent of the "there is no such thing as bad publicity" marketing plan and look where it got him...


----------



## Hxxx

If I'm a mechanic but I'm not certified nor I have the experience to carry such tasks and you know it and I've proved to be a nub, but you keep bringing your Lamborghini Gallardo or your GT-R to my house for repairing or tweaking, who's fault it is?


----------



## GelHost

Aldryic C said:


> I wouldn't call advertising your incompetence and poor security to be 'brilliant'.  Not to say I'm unappreciative of their efforts, however - we had a number of new signups last night/today from people no longer trusting teenagers, and wanting a reliable host.


 See someone is getting the profit from their brilliance.


----------



## GelHost

DomainBop said:


> Vitaly Borker is probably the most famous proponent of the "there is no such thing as bad publicity" marketing plan and look where it got him...


I just pictured GVH in that case lol, got I'm loving this soap opera.


----------



## drmike

hrr1963 said:


> c'mon, we all know drmike is privileged everywhere... He is like the Wikipedia guy


The Wikipedia guy, Wales  hahah... not quite, but I think that's a compliment 

There are a bunch of companies I have helped over the last oh year or better.  Big picture stuff, some mentoring, mostly marketing/communications, written copy (ads, brainstorming, content, contracts, etc.).

That's about all there is to it.  Gives some additional access to folks  company managers and owners, and some earned respect.


----------



## texteditor

DomainBop said:


> Vitaly Borker is probably the most famous proponent of the "there is no such thing as bad publicity" marketing plan and look where it got him...


To be fair, I _have_ now heard of his company, as it is listed just above his child pornography, fraud and threat charges.


----------



## drmike

texteditor said:


> To be fair, I _have_ now heard of his company, as it is listed just above his child pornography, fraud and threat charges.


Vitaly sure took a worse turn... Last time I heard him going off, I think he vulgarly threatened a female customer with something akin to violent assault.  Not a nice fellow.

I am meh about the "content" feds found like that.... Not saying it's untrue, just too convienent and easy to plant on people they have shakey cases against or where they feel someone isn't being punished severely enough.  An occurrence that is way too common and impossible to fight.  What's the legal defense for such?


----------



## DomainBop

drmike said:


> The Wikipedia guy, Wales  hahah... not quite, but I think *that's a compliment*


Whether it's a compliment or not depends who you ask.  Jimmy Wales has been one of the TheRegister's prime targets over the years (he's not too fond of them either).  http://www.theregister.co.uk/Tag/jimmy%20wales


----------



## drmike

Wow talk about a media fetish with one fellow.

I am aware of Wales early on investments and propensity towards certain content of the adult nature.

Surprised to see the Register full bore on-about Wales. 

Might you know the background reason @DomainBop?


----------



## DomainBop

> Might you know the background reason @DomainBop?


No idea why.


----------



## texteditor

Jimbo's a bit crazy but then he took a shit on Bitcoiners and won my respect back


----------



## DomainBop

inb4 MannDude says to keep it on topic...

Someone alerted me to this which is on topic:  when ordering a GreenValueHost VPS one of the options is:



> *One-time Linux Server Hardening Service* - Advanced Linux server hardening/security auditing service for all flavors of Linux Contact sales for more information ($10.00 USD One Time)


Maybe it's just me, but a company that doesn't even bother to harden its own billing site offering server hardening services to others for a fee seems like a ripoff. //facepalm //rolleyes


----------



## hellogoodbye

Apparently GVH is "receiving a large Layer-7 DDoS attack" at the moment.


----------



## Nett

The DNS servers are down as well


----------



## iWF-Jacob

Nett said:


> The DNS servers are down as well


Aw man! It must be a party over there!


----------



## drmike

Things *should* be back to normal now... 3-4 hours my understanding on the attack.. Layer 7 and RamNode's filtering was choking... No idea about size of attacks or anything - filtering tends to be like that unless you are high priority person/well versed with such/tied into things.

Haters are going to hate.  DDoS is what it is...  But some other mitigation was done in places and additional changes implemented to attempt to gracefully weather such.


----------



## GelHost

Ok, now I kind of feel bad for them.


----------



## Nett

> The Layer 7 DDoS attacks are getting larger and more serious. We've come up with a temporary mitigation solution. Can't post it here in case the attacker is reading this but we're going to work to get things back online ASAP.


----------



## drmike

Allegedly it is a Wordpress exploited fleet of attackers.  Attacks target a single file, but move around when blasted.

RamNode is where they were hosting their site.  RamNode's server got punched with very high load.   Nulled them.

Unnulled and doing other mitigation.

DDoS is what it is.   I am not contributing to mess or cleanups tonight.... Instead I am making popcorn and watching with everyone else.


----------



## Nett

RamNode provides 10Gbps DDoS protection only. That's *tiny* compared to real DDoS attacks.


----------



## hellogoodbye

This is getting beyond ridiculous. 

For all that people slag on GVH and its teenaged chief of operations running the joint, I think whoever that is launching this DDoS attack is the real child here. The earlier compromise (and I do consider it to be a compromise, irregardless of their own definition of the term) was already crossing the line, but at least it revealed something useful-- that GVH had failed to harden basic security with their WHMCS. This however serves absolutely no purpose besides being a dick just because he can.


----------



## drmike

Nett said:


> RamNode provides 10Gbps DDoS protection only. That's *tiny* compared to real DDoS attacks.


That is true.  Assuming they have that much filtering and connectivity dedicated to such... not making a judgement either way... cause...

What is going to happen way before then is a puny VPS is going to blow up IO on container and probably ruin the entire server.  Ending up in null, and or more properly a service suspension.


----------



## Nett

> The attack literally increased it's size by 10 in the past 30 minutes. We're still trying to find a solution.


----------



## Nett

drmike said:


> What is going to happen way before then is a puny VPS is going to blow up IO on container and probably ruin the entire server.  Ending up in null, and or more properly a service suspension.



_Just delete the VPS and close the account. /joking_


----------



## DomainBop

conspiracy theory: Jon is upset that nobody started any new GVH threads today so he's DDoSing himself to get attention. 



> The attack literally increased it's size by 10 in the past 30 minutes. We're still trying to find a solution.


My suggestion to GVH would be STFU and not give any details about the attack to anyone until the attack is completely mitigated. DDoS attackers epenis sizes (and the intensity of their attacks) tend to increase each time their victims publicly post about the attacks.


----------



## KuJoe

RamNode is using CNServers for their DDOS protection and CNServers does not offer any form of layer 7 protection so any sized attack will bypass their protection unless RamNode has something implemented on their network or their server to mitigate it.


----------



## drmike

I ran out of popcorn ... so I went back to being helpful... Ho hum...

Ramnode had like 200+ load on that server, due to attack... so Layer 7 they have, meh... what shall I call it?

People diss CloudFlare, me included, but up it went. Things were alright.  Then CF started straight passing traffic. Growl.

So CF back in place now.... and more heavy duty linebacker blocks put up... Whee...

http://www.downforeveryoneorjustme.com/greenvaluehost.com

Let's see how long this magic carpet ride lasts.

Done offering free mob protection for tonight.


----------



## MannDude

drmike said:


> I ran out of popcorn ... so I went back to being helpful... Ho hum...
> 
> Ramnode had like 200+ load on that server, due to attack... so Layer 7 they have, meh... what shall I call it?
> 
> People diss CloudFlare, me included, but up it went. Things were alright.  Then CF started straight passing traffic. Growl.
> 
> So CF back in place now.... and more heavy duty linebacker blocks put up... Whee...
> 
> http://www.downforeveryoneorjustme.com/greenvaluehost.com
> 
> Let's see how long this magic carpet ride lasts.
> 
> Done offering free mob protection for tonight.


Were they using free or paid CloudFlare? Free or the $25/mo plan is useless against attacks. And if they didn't change their IPs then CF won't be of any use anyhow.


----------



## Nett

Layer 7 DDoS attack protection is only available for Business ($200/month) and Enterprise (~$5000/month) plans.

https://www.cloudflare.com/plans


----------



## drmike

"It's not just you! http://greenvaluehost.com looks down from here."

This is why you have staff around the clock or get your lazy princess ass out of bed, or never go to bed when someone is burning your castle down.

Kids though, they have a bed time or they might get grounded and their i-devices untethered.


----------



## Nett

LOL. They switched to CloudFlare and it's still down


----------



## drmike

MannDude said:


> Were they using free or paid CloudFlare? Free or the $25/mo plan is useless against attacks. And if they didn't change their IPs then CF won't be of any use anyhow.


Well they were to start with base CF.. and tier up appropriately....  Much of this botnet would be pre-bad-listed with CF and get captcha'd by CF or dealt with otherwise by them.   The botnet is still running on compromised machines that were put on lists nearly 2 months ago.

As for the IP previously exposed, unsure what they did there.  I run nested onion-like layers with throwaway front end IPs for this reason. GVH does not.

I'll say it once, this is the type of stuff that seperates the boys from the men - the real providers from the hobby hosts.  Other companies I know / deal with / etc. run monitoring of everything.  An outage like this forces someone with some ability out of bed even on Christmas.  An hour into this and other staff gets woke up and policies are in place for what to do (and if not, people are just that damn good).

There are plenty of inept companies of all sizes that would fail like this.  So not grinding GVH.   People sleeping during stuff like this with crap ongoing, I have no love for it. Impacted customers shouldn't be sunshine, rainbows and daisies either about it.


----------



## Hxxx

drmike is this DDoS just to their client area or their whole networks, including VPS and other stuff?

I though this was just to the WHMCS... no client affected. Is it otherwise?


----------



## KuJoe

drmike said:


> This is why you have staff around the clock or get your lazy princess ass out of bed, or never go to bed when someone is burning your castle down.
> 
> Kids though, they have a bed time or they might get grounded and their i-devices untethered.


Actually the best thing they can do is go to bed and worry about it later. They can either throw money at the problem or wait for the attacker to get bored, and waiting doesn't cost a dime. Sure, they can't take any new sales while their website is down but if the trade-off is not having to pay costly mitigation fees and bandwidth overages then so be it.

Now if this truly is a layer 7 attack then there are much cheaper mitigation methods out there than CloudFlare that are much more effective.


----------



## drmike

True to some extent Mr. KuJoe!

Problem is, since I went to looking, is that ability for customers to log into panels seems to be hosed also.  WHMCS appears down along with their website.

Someone with services now or prior can maybe confirm if other direct URLs to Solus are also down.


----------



## Nett

SolusVM is working fine.

https://solusvm.securenetworkpanel.com/login.php

https://solusvm.secureserverpanel.com/login.php


----------



## drmike

Nett said:


> SolusVM is working fine.
> 
> https://solusvm.securenetworkpanel.com/login.php
> 
> https://solusvm.secureserverpanel.com/login.php


Thank you... hopefully people have such handy....

Their SSL certificate is fubared...

-------------------------------

This Connection is Untrusted


solusvm.securenetworkpanel.com uses an invalid security certificate. The certificate is only valid for solusvm.secureserverpanel.com (Error code: ssl_error_bad_cert_domain)


----------



## Nett

The first URL is their old URL and the second is their new one. I am surprised that they "don't have time" to remove the old URL.


----------



## WSWD

drmike said:


> Kids though, they have a bed time or they might get grounded and their i-devices untethered.


Well on the plus side, it's not a school night, so hopefully no homework to do, and maybe the parents will extend their bedtime slightly.  :lol:


----------



## Nett

Looks like RamNode suspended GVH's account. Ping times out.


----------



## Nett

Got 403 error when accessing the cart.php.


----------



## DomainBop

GVH WHMCS:



> Site error: the file */home/securegr/public_html/index.php* requires the ionCube PHP Loader ioncube_loader_lin_4.4.so to be installed by the website operator. If you are the website operator please use the ionCube Loader Wizard to assist with installation.





> *Critical Error*
> Could not connect to the database


*I love circus clowns!*

tip #1: http://www.ringling.com/ClownAuditions/

tip #2: GVH should stop offering managed services and "server hardening" and issue refunds to all the poor customers they conned into buying managed services and server hardening (after they offer pro-rated refunds to the shared/reseller customers they threw under the bus).

tip #3: Visit WHT and reread the countless words of advice that people have given you over the past 2 years. If the reason you haven't followed any of the advice is because reading comprehension isn't one of your strong points then you might want to work on improving your reading comprehension skills with these convenient English (your favorite!) language worksheets before you reread the advice on WHT.


----------



## Tactical

/home/securegr/ lol it ain't secure!


----------



## MartinD

Way off topic and now turned in to the usual shitfest. Game over.


----------

