# ksoftirqdx



## Nick_A (Apr 14, 2014)

Anyone seeing a ton of ksoftirdqx processes lately? Might be linked to hacked zpanel instances, though we haven't confirmed yet. We are shutting down/suspending people left and right for this process running over 8 CPU load consistently. It always seems to run out of /tmp:

http://gyazo.com/06f3e063251830336afc7b8c3791fbd6


----------



## Hxxx (Apr 14, 2014)

Dang, you guys ahead of anything ^ ^ .

Any other sign as of why is suspicious other than the tmp stuff.


----------



## MartinD (Apr 14, 2014)

Nick_A said:


> Anyone seeing a ton of ksoftirdqx processes lately? Might be linked to hacked zpanel instances, though we haven't confirmed yet. We are shutting down/suspending people left and right for this process running over 8 CPU load consistently. It always seems to run out of /tmp:
> 
> http://gyazo.com/06f3e063251830336afc7b8c3791fbd6


Yes and it must be due to zpanel. I haven't seen it anywhere else other than VM's with zpanel running.


----------



## Nick_A (Apr 14, 2014)

hrr1963 said:


> Dang, you guys ahead of anything ^ ^ .
> 
> Any other sign as of why is suspicious other than the tmp stuff.


It's always the same pattern of abuse. 8 core load, runs out of that subdirectory in /tmp, etc. Seeing it on all three networks, and really only recently.

Thanks @MartinD.


----------



## kaniini (Apr 14, 2014)

Why not set up your containers where /tmp is mounted noexec?


----------



## perennate (Apr 14, 2014)

Looks like my VM with End of Reality got hacked, was running latest zpanel, I'm switching to vestacp and reformatting :/

Luckily nothing important running on it.


----------



## wlanboy (Apr 14, 2014)

Not zpanel again...


----------



## MartinD (Apr 14, 2014)

wlanboy said:


> Not zpanel again...


Not even sure it's 'again' - just same old :|


----------



## Magiobiwan (Apr 14, 2014)

I've been seeing this but hadn't made the connection. Oh fun. Time to add some entries to my personal abuse-check scripts...


----------



## MartinD (Apr 14, 2014)

There's also ".IptAbleS" and similar popping up, too. Found in /boot


----------



## Wintereise (Apr 14, 2014)

Just nuke it and move on, imo -- people who outlawed it in the ToS the first time around took the best decision.


----------



## perennate (Apr 14, 2014)

Yes, ban OpenSSL too


----------



## wlanboy (Apr 14, 2014)

perennate said:


> Yes, ban OpenSSL too


In my personal view the OpenSSL team handled security issues in a quite different way than the zpanel team.

Hours vs months...


----------



## alexmocanu (Apr 20, 2014)

Did one of you find a solution other then moving to another panel ? 

 

In my case, moving is not an option.. 

 

Same thing, server loads at max, ksoftirqdx running from apache, i am killing the process but the problem is that ksoftirqdx   was back running again, after 4-5 hours.

 

The odd thing is that i have 2 vm's running zpanel, but the both got the same problem, running ksoftirqdx, at the same hour, minute. 

 

Any idea ?


----------



## Me.B (Apr 20, 2014)

wlanboy said:


> In my personal view the OpenSSL team handled security issues in a quite different way than the zpanel team.
> 
> Hours vs months...



Sir,

I'm a member of Zpanel team, your claim is totally out of the context. you talk about zpanel taking month's to fix any flaw.


1. The project is totally open source so any one could fix flaws reported.


2. Could you check the announcements in our forum? You will see we issue quick security fix before implementing it in the release. Zpanel even display now a security news.


Most of the trashing here is over Zpanel 10.0.2 and even that we issued fixes within 2 days far from all the bashing it took month's.


I'm new to the team since 3 month's and what I saw, we rush for checking flaws and see again the announcement we had some security fixes most of them were related to third party code we used ( roundcube/pchart!).


So if you have any issue over zpanel you are welcome to report it and happy to try to figure out this.

Any project will have issues and we are eager to fix any if reported. 


M B


----------



## Me.B (Apr 20, 2014)

alexmocanu said:


> Did one of you find a solution other then moving to another panel ?
> 
> 
> 
> ...


I already replied in zpanel forum

http://forums.zpanelcp.com/Thread-ksoftirqdx-apache-service-loads-server-for-no-reason

And offered you to check my self your server. So send me server access and I will check if you had any issues in zpanel.

We are always taking seriously security reports and will do our best to fix any flaw in zpanel. IF it's the case. I still see you have more odds to get problems from WP or any other CMS here.

Notice your zpanel is not updated AS advices and we released the latest fixes over a moth ago.


M B


----------



## HalfEatenPie (Apr 20, 2014)

Me.B said:


> Sir,
> 
> I'm a member of Zpanel team, your claim is totally out of the context. you talk about zpanel taking month's to fix any flaw.
> 
> ...



You do realize it took them more than two days right?  If I recall correctly joepie contacted the project head-individual a week or so in advance (maybe more?) about the issue.  Was basically told that there was no issue with it.  

Then a week later the entire site was compromised.  

Anyways welcome to the forum


----------



## jarland (Apr 20, 2014)

Yep. It's definitely zpanel no question about it. Whether it's specific to their code or software versions they install I don't know but I have only seen this with zpanel installed.


----------



## Me.B (Apr 20, 2014)

jarland said:


> Yep. It's definitely zpanel no question about it. Whether it's specific to their code or software versions they install I don't know but I have only seen this with zpanel installed.


Again no argument... It's just zpanel.... Did you check the bug fixes since then?


----------



## Me.B (Apr 20, 2014)

HalfEatenPie said:


> You do realize it took them more than two days right?  If I recall correctly joepie contacted the project head-individual a week or so in advance (maybe more?) about the issue.  Was basically told that there was no issue with it.
> 
> 
> Then a week later the entire site was compromised.
> ...



Do you realize that you keep rolling an old story that since then we did 2 releases? And I'm in the team since only 3 month's, using it on my own servers and won't EVER accept that security is taken not seriously.


Now you talk about zpanel servers taken down. Are you aware that the severs were taken down when the admin saw that some account got compromised? It was precaution. And that the mighty joepie took over another server not running zpanel but using brute force? That helped him later gain control over the forum?

This is not the first issue an open source project face. Who remember kloxo? Or even check Plesk CVE? Phpmyadmin! Roundcube.


All I can say if you have any security issues I will do my best to escalate or fix it. And latest we got I saw the fix rolling in 24h! Just check the announcement sections.


Zpanel is open source for the community and everybody is welcome to improve or fork it and it's on github now.

M B


----------



## HalfEatenPie (Apr 20, 2014)

Me.B said:


> Do you realize that you keep rolling an old story that since then we did 2 releases? And I'm in the team since only 3 month's, using it on my own servers and won't EVER accept that security is taken not seriously.
> 
> 
> Now you talk about zpanel servers taken down. Are you aware that the severs were taken down when the admin saw that some account got compromised? It was precaution. And that the mighty joepie took over another server not running zpanel but using brute force? That helped him later gain control over the forum?
> ...


My point was not specifically on the issue of compromised services but mostly the response the "project coordinators" had.  From my perspective they simply shrugged it off instead of even attempting to investigate it and later (once it became even more public knowledge) it was exploited.  You advertise yourself as a FOSS project.  Awesome!  You advertise that your software can be used in a more high-priority environment.  But, does not respond well to security concerns when they were initially brought up.  Not awesome.  

zPanel can change.  Sure.  It probably has changed.  Awesome.  But so far it hasn't proven (at least to me) the way they addressed problems.  Hopefully you joining the team has improved this greatly.


----------



## jarland (Apr 20, 2014)

Me.B said:


> Again no argument... It's just zpanel.... Did you check the bug fixes since then?


Nope, I don't use it.


----------



## wlanboy (Apr 20, 2014)

Me.B said:


> Sir,
> 
> I'm a member of Zpanel team, your claim is totally out of the context. you talk about zpanel taking month's to fix any flaw.
> 
> ...


I did run two zpanel instances for months and I migrated each one to VestaCP.

Right out of the reason how (at that time) people which are responsible for the project handle things.

Like yourself.

Zpanel is superb, best of the best and everyone who is telling something about problems is a liar and trasher.

But I will stop saying anything of zpanel - you just confirmed my decision.

I will never use zpanel again.


----------



## Me.B (Apr 20, 2014)

wlanboy said:


> I did run two zpanel instances for months and I migrated each one to VestaCP.
> 
> Right out of the reason how (at that time) people which are responsible for the project handle things.
> 
> ...


I'm new to Zpanel TEAM but using zpanel since over a year.


Sir great for you if vestaCP fits your need or Cpanel or Plesk.

All my statement didn't say that zpanel is perfect but that we care for security and all comments are welcome so we can improve the product, so don't twist my words here. 

Zpanel still have many bugs to be fixed and many features that must be plugged in. I feel more and more that zpanel trashing is getting personal. It's not about the project it self, it's turning over the team. Did you saw my replies in zpanel support forum before? I've just used zpanel and all I tried to do when joining zpanel is helping improving the product and ensure also that security is OK, not do like so many just bashing the product without using it or following it.


So if any had solid arguments over zpanel security I'm happy to hear them and forward them. If it's an ego problem/personal I can't fix it.


I'm just trying to figure out what's wrong in Zpanel so I fix it at least for myself...


----------



## Me.B (Apr 20, 2014)

HalfEatenPie said:


> My point was not specifically on the issue of compromised services but mostly the response the "project coordinators" had.  From my perspective they simply shrugged it off instead of even attempting to investigate it and later (once it became even more public knowledge) it was exploited.  You advertise yourself as a FOSS project.  Awesome!  You advertise that your software can be used in a more high-priority environment.  But, does not respond well to security concerns when they were initially brought up.  Not awesome.
> 
> zPanel can change.  Sure.  It probably has changed.  Awesome.  But so far it hasn't proven (at least to me) the way they addressed problems.  Hopefully you joining the team has improved this greatly.


So check here:

http://forums.zpanelcp.com/Forum-News-Announcements--36

I don't have any gain from cheating here... I don't care for my ego as I'm not the main developer here but mainly more a user.


We got a report over pchart bug 

http://www.pchart.net/advisory

A zpanel user pointed it could lead to RCE. I can ensure you 24h hours later the infos led to this announcement:

http://forums.zpanelcp.com/Thread-Pcharts-Urgent-Vulnerability-Fix

This happened 21 Feb, It triggered an internal discussion over reviewing third party software in zpanel and thus rouncube that had too a bug, so we need to update it:

Issue 4 days later another announcement:

http://forums.zpanelcp.com/Thread-RoundCube-Urgent-Security-HotFix

And  a week later we pushed 10.1.1 after we rushed for testing. As it was not only a security fix but had some minor fixes that needed to be tested.


This is how I saw zpanel team working and you don't have an idea how much time we need to test the software or in support.


So if you think it's not enough, I will be happy to hear how it can be improved further, notice the problem that most users too love Free FOSS and don't try to spend time on it. It would be great if we get more help from security experts unhappy over zpanel security. That would improve the problem and may be in the future we gain back trust.


----------



## Patrick (Apr 21, 2014)

You just linked to 3 posts, you only allow 3 post views per IP. That's just stupid considering we have to register to a forum to view patches because you guys couldn't be bothered to test before releasing updates.
 



> You've exceeded the maximum number of posts (3) you can view as a guest. To remove this message and become a member please register a free account. It will only take a few moments and you'll be able to view posts normally.


----------



## Me.B (Apr 22, 2014)

Patrick said:


> You just linked to 3 posts, you only allow 3 post views per IP. That's just stupid considering we have to register to a forum to view patches because you guys couldn't be bothered to test before releasing updates.


It would be a very good point if you were really using Zpanel not just looking for argument to say those stupid guys don't take security seriously!

In Zpanel you have already a module zpanel news reading RSS from announcement section. So you will see the new annoncement then you would go to the forum reading. Indeed there is a 3 posts view limitation and then? Registration is free and will allow you to ask for free support.  I saw other panels requiring registration before downloading and no one called them stupid. I would say it's annoying. But you are not barred from reading headlines.


Notice on facebook zpanel announcements are open... And I back a blog for announcement.


You argument have some ground over security sections only. And I will forward it to the team.

so thanks @Patrick


M B


----------



## Me.B (Apr 22, 2014)

See here how Zpanel is trashed with bad faith:

http://www.liatsisfotis.com/2014/01/multiple-vulnerabilities-in-zpanel-1002.html

Post date 1/January while he claim this got unpatched for 10 month's until 10.1.1.

That's totally wrong as we got before 10.1.0 released 4-8-2013 ! 

While emergency patch released 2-4-2013 

http://forums.zpanelcp.com/Thread-ZPanel-HotFix-Please-ensure-you-apply

Notice his first post over zpanel 10.0.2 was:

http://www.petrosandreou.com/2013/07/multiple-vulnerabilities-in-zpanel-1002.html

4 July ...

Now backing it took month's is totally exagerated while he posted over the flaw after zpanel made it public!


----------



## HalfEatenPie (Apr 22, 2014)

@Me.B

Relax. I understand you really enjoy being part of the ZPanel team and I understand for you it's a project to get behind. Unfortunately for myself (and a few people on here) it's not. My opinion on ZPanel is basically "never using it again" especially with the way the project lead responded (obvious paraphrasing but "It's an enterprise-level software" to "contribute it yourself"). If I recall ZPanel's theming system still uses (or used to) PHP's EXEC command (I don't know if they've fixed this yet nor have I actually actively checked) which is a huge no-no. Actions like these makes me lose faith in a development team and questions what other major mistakes have they done that we haven't caught.

People make mistakes. It's a given fact. We're all human. But the way the project head has responded to some answers even before the hacking incident shows me that I can't put my support behind it.


----------



## peterw (Apr 22, 2014)

You need to spend a lot of time until people trust the zpanel team again. There are enough other panel so I will not use zpanel again.


----------



## Me.B (Apr 22, 2014)

HalfEatenPie said:


> @Me.B
> 
> 
> Relax. I understand you really enjoy being part of the ZPanel team and I understand for you it's a project to get behind. Unfortunately for myself (and a few people on here) it's not. My opinion on ZPanel is basically "never using it again" especially with the way the project lead responded (obvious paraphrasing but "It's an enterprise-level software" to "contribute it yourself"). If I recall ZPanel's theming system still uses (or used to) PHP's EXEC command (I don't know if they've fixed this yet nor have I actually actively checked) which is a huge no-no. Actions like these makes me lose faith in a development team and questions what other major mistakes have they done that we haven't caught.
> ...


Thanks. But to make it clear the old nag over zpanel theming system is totally out of context.


1. Themes now use bootstrap and no one can add them thru the panel you should go manually and upload the files.


2. ALL The panel have phpexec enabled, AS it's a panel and need to execute external command to get things done. How do you expect a panel to work without phpexec?

3. The row was that that themes could use php exec it's a total non sense man. Check zpanel code and you will see how it works. Themes are unlike WP or such it's only admins that handle them first. Like many other features. When I hear this blame over themes I feel it's totally not having any ground for any one understanding how zpanel works and I see why the leader got frustrated over this row. He tried to explain that themes are meant to be manipulated only by admin while "the hacker" was pushing over and over, you have a flaw. I can do anything with them. Indeed with zpanel admin rights I can upload too modules that can take over the whole server too... Hope you see the non sense of the claim. Zpanel had more serious issues that was not reported then over LFI exploit, as the team annonced it and fixed it.

See here a report zpanel website hacked!!

http://forums.zpanelcp.com/Thread-My-zPanel-is-hacked-Files-have-removed

I checked my self the server, he got CMS + phpBB modded.... And found nothing 

So the problem with zpanel bad press we made the headlines and no one checking for the real story and check the facts. I use zpanel and looked for facts before moving in, so I care over security.


M B


----------



## Me.B (Apr 22, 2014)

And to be a bit rude here. I don't care if you want to use Zpanel or not. It's not the issue.

I just want facts and security reports/advises from experts over what we could improve in security or what we missed so we can beef up security. All those I confronted bashing zpanel none had pointed me a flaw, all was "it's on the news zpanel is not secure".


----------



## HalfEatenPie (Apr 22, 2014)

Me.B said:


> And to be a bit rude here. I don't care if you want to use Zpanel or not. It's not the issue.
> 
> I just want facts and security reports/advises from experts over what we could improve in security or what we missed so we can beef up security. All those I confronted bashing zpanel none had pointed me a flaw, all was "it's on the news zpanel is not secure".


I don't think you're getting my point.



> ZPanel is an easy to use, enterprise class web hosting control panel with support for unlimited resellers. From the largest business to SOHO or development environments, ZPanel can support your needs.
> 
> Source: http://www.zpanelcp.com/about/features/





Me.B said:


> 1. The project is totally open source so any one could fix flaws reported.


(This basically an indirect way of saying "We call ZPanel an enterprise class solution but if a problem happens you can fix it yourself")



HalfEatenPie said:


> *My opinion on ZPanel is basically "never using it again" especially with the way the project lead responded (obvious paraphrasing but "It's an enterprise-level software" to "contribute it yourself")... Actions like these makes me lose faith in a development team and questions what other major mistakes have they done that we haven't caught.*
> 
> *People make mistakes. It's a given fact. We're all human. But the way the project head has responded to some answers even before the hacking incident shows me that I can't put my support behind it.*


Of course scripts should use EXEC. But EXEC should NEVER be used in a theme for WHATEVER reason. It's called Risk Management (i've already linked this a ton even on this forum alone) and the development team should be working to minimize this.

Also, ignoring the EXEC problem. My problem is with the *Project Lead and his responses.* People screw up. Solus has screwed up, WHMCS has screwed up, everyone screws up. The important part is the response that comes with it. I feel like the Project Lead was way too... I can't put a word to it, but just used words whenever it was convenient for him.

*Edit:* And I'm going to drop this conversation (at least my part for now) here. I feel like it's going in circles.

*Edit 2:* Fixed a few things like a few misspellings and added the quote from the site + previous post.


----------



## HalfEatenPie (Apr 22, 2014)

Me.B said:


> And to be a bit rude here. I don't care if you want to use Zpanel or not. It's not the issue.
> 
> I just want facts and security reports/advises from experts over what we could improve in security or what we missed so we can beef up security. All those I confronted bashing zpanel none had pointed me a flaw, all was "it's on the news zpanel is not secure".


Also to be on the same level as this quote (my apologies for being rude right here, and I just broke my previous edit's statement haha). I don't care if you support ZPanel or not. What I care about is my clients running possibly free vulnerable software that gets hacked and our services get utilized to perform an attack on an innocent individual (or basically be used for negative purposes). Yes we have systems in place to minimize this but again, you can't be too careful. This is again Risk Management right here.


----------



## Me.B (Apr 22, 2014)

HalfEatenPie said:


> Also to be on the same level as this quote (my apologies for being rude right here, and I just broke my previous edit's statement haha). I don't care if you support ZPanel or not. What I care about is my clients running possibly free vulnerable software that gets hacked and our services get utilized to perform an attack on an innocent individual (or basically be used for negative purposes). Yes we have systems in place to minimize this but again, you can't be too careful. This is again Risk Management right here.


Great here at least we agree. BUT you must too agree WP/ Joomla/PHPbb and all alike are a big mess, especially when you get them with newbies that won't update anything. I'm in hosting biz since over a decade and I still fight with this CMS mess all the day, had to shutdown customers all the time over that. I can't ban them using WP / Joomla so I ended up helping them hardening their solution or trying to reduce surface attack. I do the same over zpanel, as I change some default settings. So despite you don't think Zpanel so good your users might still use it. So either you could help us, if you notice anything we should fix. 

Even cpanel with newbies can turn into a mess and this the big problem in VPS users, I see a lot moving from a shared managed service to self managed VPS like they have a magic wand to admin all servers issues.


So and to show again how this discussion is out of the scope of the first post no one looked for the origin of the problem reported it was only bashing zpanel lack of security, while we managed to look over this to find this issue not related to zpanel but ubuntu IRQS:

http://askubuntu.com/questions/7858/why-is-ksoftirqd-0-process-using-all-of-my-cpu

This didn't prevent me from taking some feedback over security annoncement might be in a Blog alike not in forum and I'm currently checking this ridiculous theme "non issue" so we lock it down, so we move into serious talk as some are still not getting it and don't want to check how.


M B


----------



## Patrick (Apr 22, 2014)

Me.B said:


> It would be a very good point if you were really using Zpanel not just looking for argument to say those stupid guys don't take security seriously!
> 
> You argument have some ground over security sections only. And I will forward it to the team.
> 
> ...


My clients unfortunately use it and we have to suspended because of the named process in the thread, I can't give them a actual url to the fix because of your limit and i'm not going to register to your useless forum which god knows when will be hacked again.

I just give them alternatives which have actual care and time taken into and help them rebuild their VM from scratch with Vesta etc.


----------



## Me.B (Apr 22, 2014)

@HalfEatenPie Got your point. Testing currently sandboxing ALL theme folder in lower permissions. 

And I've been thinking my self about sandboxing more stuff as in my own setup I don't run webmail on zpanel host, I will check it and might submit this to the team so we make some changes.

Adding preventive layers has never been bad idea's but you must notice this had been presented AS A FLAW! I could then flag a lot a flaw of this type in many products we use daily... Any way let's see where my own test would lead.

This doesn't mean Zpanel is unsecure man too! It's unsecure when you have a flaw that lead to a hack, we shipped Roundcube that had RCE grade flaw and no one talking about it, while if talk about risk management we should talk about risk evaluation and here roundcube flaw was a major threat while themes were minimal.

M B


----------



## Me.B (Apr 22, 2014)

Patrick said:


> My clients unfortunately use it and we have to suspended become of the named process in the thread, I can't give them a actual url to the fix because of your limit and i'm not going to register to your useless forum which god knows when will be hacked again.
> 
> I just give them alternatives which have actual care and time taken into and help them rebuild their VM from scratch with Vesta etc.


 Forum was not directly hacked but they gained access over a user server first. Notice the zpanel team didn't build the forum as this totally hilarous! We were using VBulletin and now myBB as you will notice and no hacks.

Here I paste the statement and at least you could add to the list zpanel team are liars:



> *ZPanelCP Server has not been compromised!*
> 
> After many allegations that our community forums / website have been compromised we can safely announce this is false information.
> 
> ...


 http://forums.zpanelcp.com/Thread-ZPanelCP-Server-has-not-been-compromised

Zpanel 10.0.2 got an LFI exploit targetted with a bot. Strange that many users are still running this release over a year old while we had since 2 releases and some hotfixes!

If you won't go to our forum. I've come here so post the list or PM me. I will be happy to see if we can fix more stuff. I never asked you to turn to our forum.

So.. Good luck if vesta works for your customers but if you could help us fix zpanel or report issues we don't see, so let's see. @


----------



## HalfEatenPie (Apr 22, 2014)

Patrick said:


> My clients unfortunately use it and we have to suspended become of the named process in the thread, I can't give them a actual url to the fix because of your limit and i'm not going to register to your useless forum which god knows when will be hacked again.
> 
> I just give them alternatives which have actual care and time taken into and help them rebuild their VM from scratch with Vesta etc.


Just like this.

Unlike Joomla or Wordpress (by the way, we have kicked clients out due to their unsecure Joomla), ZPanel isn't widely accepted or used. An alternative is suggested (some hosts even prohibit software such as Kloxo and/or ZPanel) and (when time is available) we help them migrate away to different control panels.



Me.B said:


> @HalfEatenPie
> 
> Adding preventive layers has never been bad idea's but you must notice this had been presented AS A FLAW! I could then flag a lot a flaw of this type in many products we use daily... Any way let's see where my own test would lead.


Just because other software have the possibility for the flaw doesn't me your standards should accept it. And again you seem to be missing the point of my comments still.



HalfEatenPie said:


> I don't think you're getting my point.
> 
> My problem is with the *Project Lead and his responses.* People screw up. Solus has screwed up, WHMCS has screwed up, everyone screws up. *The important part is the response that comes with it.*


I'm not questioning the security of the bundle of softwares you utilize but the response your team gives to others.



Me.B said:


> This doesn't mean Zpanel is unsecure man too! It's unsecure when you have a flaw that lead to a hack, we shipped Roundcube that had RCE grade flaw and no one talking about it, while if talk about risk management we should talk about risk evaluation and here roundcube flaw was a major threat while themes were minimal.


I understand the point you're making (by the way, for clarification, part of the Risk Management assessment includes Risk Evaluation) with that statement and of course it's difficult to code something that's 100% fool-proof. I mean we all strive to prevent as much as we can but in the end some things we may have overseen might have slipped past us and got shipped. *What I'm talking about is the response the team gives before the issues even start.* You indirectly tell these people to patch it themselves and to submit it. You do realize a major portion of these individuals don't know how to code and (even worse) barely understand how to setup the panel (know the bare minimum about server hardening).

There's issues in all panels of course, but from my own (and many other individuals') evaluations we've come to recognize some of the alternatives to ZPanel to be far better than ZPanel itself.


----------



## Me.B (Apr 22, 2014)

@HalfEatenPie You didn't read the announcement section neither saw zpanel news module SIR.

I don't say people must patch it them self. IT would be totally idiot. I said in emergency mode you could. I saw report over roundcube problem, so I patched it. May be by patch you expect an autoupdate? Currently the update script is improving.

Last 2 patches were submitted by the team one for pchart ( user reported in forum and we immediatly fixed it in github) and another in internal review over roundcube.

I submitted later a request to add robot.txt banning zpanel indexing as preventive measure, and working on how to ban zpanel from defaulting on server IP. Also submitted masking smtp banner signature and same over webserver and would try my best over many other direction. And this not only me other team members submitted other "preventive" patches...

IF it's not enough, ok. The problem you say zpanel is insecure? So I can say that this is not a flaw in Zpanel but the you think we work?


----------



## Nick_A (May 3, 2014)

Well, it's changed to "auditd" now.


----------



## MartinD (May 3, 2014)

Nick_A said:


> Well, it's changed to "auditd" now.


Was gonna post the exact same thing


----------



## Nick_A (May 3, 2014)

I think we're at 10 shutdowns and counting this morning.


----------



## MartinD (May 3, 2014)

Are you telling the customers and giving them info or..?


----------



## Nick_A (May 3, 2014)

Yeah same info as previous hack. Considering banning zpanel.


----------



## MartinD (May 3, 2014)

I'm edging towards the same. Just not worth the hassle at all.


Can lead a horse to water and all that


----------



## Magiobiwan (May 3, 2014)

Anyone have a hash of the binary? Next time I spot a compromised VPS I'll grab an md5 hash of the binary, but if anyone has it now, that would be useful. Still out of /tmp I assume?


----------



## MartinD (May 3, 2014)

Yeah, still from /tmp


----------



## serverian (May 3, 2014)

#!/bin/bash

containers=$(ls /vz/private/)
for CTID in $containers
do
if [ -d /vz/private/$CTID/etc/zpanel/ ]; then
echo "VM: $CTID running zPanel"
# avoid too many arguments error
MOUNTED=$(vzctl exec $CTID cat /proc/mounts | grep /tmp)
if [ -z "$MOUNTED" ]; then
echo "VM ID: $CTID is running zpanel and tmp is not secured. Wanna secure it? (y/n)"
read ANSWER
if [ $ANSWER == "y" ]; then
echo "Done on VM ID: $CTID"
vzctl exec $CTID rm -rf /tmp
vzctl exec $CTID mkdir -p /tmp
vzctl exec $CTID echo none /tmp tmpfs nodev,nosuid,noexec 0 0 >> /etc/fstab
vzctl exec $CTID mount /tmp
fi
fi
fi
done
Run on the nodes and done! No need to suspend those poor bastards!

Credits goes to @Zen


----------



## Me.B (May 5, 2014)

Nick_A said:


> I think we're at 10 shutdowns and counting this morning.


Hi,

Could check zpanel they are running. The reports we got it try to hack old zpanel 10.1.0 that we released security patch.

Zpanel used pchart2 lib that had 0 day flaw. So we updated zpanel. And seem now hackers 2 month's later use our security notice to hack zpanel again.

I would apreciate if you have any infos logs. You can pm me if it requires privacy.

M B


----------



## Me.B (May 5, 2014)

This is custom enhanced .htaccess, you should advice zpanel users to set and this will limit much of the possible damage.
 



> RewriteEngine on
> RewriteRule ^api/([^/\.]+)/?$ bin/api.php?m=$1 [L]
> RewriteRule ^apps/([^/\.]+)/?$ etc/apps/$1 [L]
> 
> ...


Also roundcube shipping with 10.1.0 have an RCE so it need to be updated.

M B


----------



## Me.B (May 5, 2014)

serverian said:


> #!/bin/bash
> 
> containers=$(ls /vz/private/)
> for CTID in $containers
> ...


Great hack but I can provide you with paths to delete if you want.

Or more interesting ways to check zpanel version.

Notice zpanel use also those temp directories:

/var/zpanel/temp

 

and 

 

/etc/zpanel/panel/etc/tmp

 

M B


----------



## MartinD (May 9, 2014)

Changed again....

"pxinit"

exe ->  (deleted)/dev/shm/40A/work/pxinit


----------



## SkylarM (May 9, 2014)

MartinD said:


> Changed again....
> 
> "pxinit"
> 
> exe ->  (deleted)/dev/shm/40A/work/pxinit


Just woke up to about 20 or so containers running the lovely auditd .ICE-unixx.


----------



## Me.B (May 11, 2014)

Take care /tmp permission is bringing down mysql for zpanel users. 

Got yesterday many reports over that and users only solution is flipping back permissions. We need to collaborate:

See here:

http://forums.zpanelcp.com/Thread-Fatal-error-Uncaught-exception-PDOException-with-message-SQLSTATE-42000

http://forums.zpanelcp.com/Thread-SOLVE-MySQL-Error-0100-Unable-to-connect-unable-to-write-tmp

twice /tmp issue yesterday.

M B


----------



## Me.B (May 11, 2014)

The main problem is over mysql. so avoid breaking it we might change default mysql temp folder to /var/temp.

This would allow 666 /tmp without side effects.

Did you notice guys the user running the process? We could ban apache executing there too?

We are trying our best to help over this.

M B


----------



## MartinD (May 11, 2014)

Or... just fix it.


----------



## WebSearchingPro (May 17, 2014)

Not sure if anyone brought it up yet, but these processes are actually bitcoin mining software in the form of malware. 

Tracked the traffic back to a central stratum mining address.

It deletes itself after it runs and runs in memory to prevent deconstructing it.


----------



## Me.B (May 19, 2014)

MartinD said:


> Or... just fix it.


BUT it's already fixed since month's !! this affect the old release. You always react like we don't issue patches.

I just pointed the limit of the current solution.


Replacing with a sed the temp directory in /etc/my.cnf will avoid customers troubles and save your time dealing with ranting customers.


M B


----------



## MartinD (May 19, 2014)

Have you told customers that your product has been updated to resolve an issue..and what that issue is?


----------

