# Simple server monitoring - Feedback appreciated (Public beta)



## NQ-Joe (Jan 11, 2014)

Howdy, I am a long time lurker since the beginning of LEB/LET (respectively VPSBoard) and finally signed up to take part in the community. My first post however has some ulterior motives and I am thankful MannDude and MartinD allowed me to post this right off the start.

 

Over the last few months I was monitoring my servers with a rather simple bash script and decided in the beginning of December to create a little project out of it. Now I know many of you are not really fond of hosted software (SaaS in fancy terms), nevertheless would I really appreciate it if you could give it a try or at least share your opinion. I myself consider the monitored data as much less sensitive and unidentifiable compared to other services out there. If you however have suggestions on how to generate more trust and security, please do tell.

 

The service hopes to aim at people who operate small groups of servers (i.e. freelancing designers/developers like me) for whom something like Nagios is just overkill. If you are interested just give it a try - I would really appreciate it.

 

Site: https://nodequery.com

Github: https://github.com/nodequery/nq-agent

Twitter: https://twitter.com/nodequery

 

Thanks for reading, looking forward to your feedback


----------



## seraphkz (Jan 11, 2014)

Interested.

Still haven't received E-mail confirmation though.


----------



## yolo (Jan 11, 2014)

I signed up and I really really really really really like this!


----------



## telephone (Jan 11, 2014)

A few things on your installer/agent:


Why am I giving you root access for simple metrics?
Why are you using 'root' instead of creating a 'nodequery' user? (with locked down privileges)
Add active network connections (through netstat)
​​Segregate connections per Port
Include active SSH connections

A couple things on your website:


Your website itself isn't mobile friendly
If the Agent is outdated do you alert the user via the CP?
​​Having users manually update the agent may be tedious for 20+ servers
User should be able to update the Agent via the CP... E.g. Users can enable "auto-update"


Overall I like the idea, but the extent of the information provided is a bit too basic for my tastes... I agree that there's a gap between needing simple statistics and Nagios/Observium, but if a problem were to arise, the information is too basic to help debug the issue.

E.g. I see in the CP that my CPU load spiked for 2 hours, but that's all... Is it the node or my VPS? I'd need more detailed statistics for pinpointing the problem. (IO Wait, Steal %, Disk IO, Network connections, etc).

 
What I'd suggest is moving to Sar (Systat). You can still display basic stats by default, but at least there'll be the option to look at detailed stats if required. Some of Sar's features:


It's lightweight (I run it on most of my VPSs)
Information is logged (in case of a network outage)
You choose the interval/average (minutes) for statistics processing
Data is exportable in multiple formats (CSV, XML)
'systat' package is available on all platforms
Here's an example of what can be done: 10 Useful Sar (Sysstat) Examples


^ I've created a basic page/panel from Sar's data, it's pretty easy to work with.


----------



## mcmyhost (Jan 11, 2014)

seraphkz said:


> Interested.
> 
> Still haven't received E-mail confirmation though.


It goes to spam.


----------



## spry (Jan 12, 2014)

Welcome, Joe!

Nice project you have there. The dashboard design is a neat one. Looking forward to see more features implemented along the way.

Keep us updated.


----------



## KuJoe (Jan 12, 2014)

My verification e-mail went to spam also. Might I suggest using a service like SendGrid for delivering e-mail? We use it for sending e-mails to our clients and it works wonderfully.


----------



## NQ-Joe (Jan 12, 2014)

telephone said:


> A few things on your installer/agent:
> 
> 
> Why am I giving you root access for simple metrics?
> ...


Thank you very much for your detailed feedback. Using root in this context indeed might not be required, when I started the script I was retrieving a few stats differently and required it. I will consider changing that for newer versions. I wanted to include active connections from the beginning but ran into a few problems parsing the result correctly and then simply forgot about it - noted. For now the session count is a good indicator for SSH connections.



telephone said:


> A couple things on your website:
> 
> 
> Your website itself isn't mobile friendly
> ...


Adapting the website for mobile devices is the next step. My priority was to get a few users testing the service while I continue working on it. If in a few weeks/months it turns out everything was a bad idea, I didn't waste too much time upfront.

Regarding agent updates, yes you will be notified on the server page when a new version of the agent is available. I considered implementing an auto-update but very soon didn't like the idea. Even if you would use multiple integrity checks it still poses a high security risk if someone manages to include malicious code.

I will also have a closer look at systat, the idea of doing it 'natively' is however still my priority.



KuJoe said:


> My verification e-mail went to spam also. Might I suggest using a service like SendGrid for delivering e-mail? We use it for sending e-mails to our clients and it works wonderfully.


I was hoping a proper postfix config would be sufficient for the start. I however had plans to switch to a message transport service like Amazon SES/SNS anyway - assuming people actually want to use the service. I will have a look at a few providers and check my options, implementing should be rather quick due to MVC structure 

Many thanks to everyone who signed up so far.


----------



## telephone (Jan 12, 2014)

NQ-Joe said:


> I was hoping a proper postfix config would be sufficient for the start. I however had plans to switch to a message transport service like Amazon SES/SNS anyway - assuming people actually want to use the service. I will have a look at a few providers and check my options, implementing should be rather quick due to MVC structure


Just setup Postfix to use Mandrill SMTP for the time being. Can't go wrong with 12,000 free transactional emails a month


----------



## NQ-Joe (Jan 12, 2014)

telephone said:


> Just setup Postfix to use Mandrill SMTP for the time being. Can't go wrong with 12,000 free transactional emails a month


That sounds even easier, thank you! I actually wanted to set up DKIM and see if it helps.


----------



## blergh (Jan 12, 2014)

Nice idea, but open-source it instead.


----------



## Melon (Jan 12, 2014)

Lovely idea, currently using Uptime Robot and am very fond of the additional statistics (RAM usage, etc.). Please don't add an auto-update feature, it's a potential security risk. Looking forward to how this project goes.

Any ideas of pricing for post-beta users?

Would also like to see the shell scripts released under a free license.


----------



## NQ-Joe (Jan 12, 2014)

Melon said:


> Lovely idea, currently using Uptime Robot and am very fond of the additional statistics (RAM usage, etc.). Please don't add an auto-update feature, it's a potential security risk. Looking forward to how this project goes.
> 
> Any ideas of pricing for post-beta users?
> 
> Would also like to see the shell scripts released under a free license.


Thanks, glad you like it. I am not planning to add auto-update, as you already said the security risk is too high even when hosted on Github. I didn't think much about pricing yet and won't for a while. I am planning to run the beta for at least a few months if enough people are interested. I myself would love to see small plans starting from $5 - I'll just have to see if that will be possible.

The shell script might be released under a free license in the future - I will definitely consider it.


----------



## TruvisT (Jan 12, 2014)

blergh said:


> Nice idea, but open-source it instead.


Agreed. For all the monitoring systems we use, we use open source for several reasons. Most the times we need to modify the code for specific reasons.


----------



## NQ-Joe (Jan 12, 2014)

TruvisT said:


> Agreed. For all the monitoring systems we use, we use open source for several reasons. Most the times we need to modify the code for specific reasons.


While I very much agree with you, my intention was to create a service for users who actually don't want or don't need to get into the code. Meaning I hope to reach users who just want to have an overall look on performance of their servers and get notified if something goes south.

I am aware that a lot of you need a more detailed and complex insight of your systems, I hope however that some of you will still try it or even refer it to people you know who might be interested


----------



## NodeBytes (Jan 12, 2014)

Any plans for an external API for integration with other sites?


----------



## NQ-Joe (Jan 12, 2014)

NodeBytes said:


> Any plans for an external API for integration with other sites?


An API is definitely something I would like to implement (and hopefully will) down the road. However, for now I want to concentrate on core services like data collection and retrieval. I don't want to develop a great API only to realize later that I failed the database design.


----------



## blergh (Jan 12, 2014)

NQ-Joe said:


> While I very much agree with you, my intention was to create a service for users who actually don't want or don't need to get into the code. Meaning I hope to reach users who just want to have an overall look on performance of their servers and get notified if something goes south.
> 
> I am aware that a lot of you need a more detailed and complex insight of your systems, I hope however that some of you will still try it or even refer it to people you know who might be interested


Do both and be awesome.


----------



## NodeBytes (Jan 12, 2014)

While I like that you want to design your database right (I worked as a DBA for a while) I am going to stay away from the agent for now as I don't like any thing running in root besides what I choose. 

While on that note, eventually you might want to set up a repo so that the agent can update off the same apt/yum update as the system.


----------



## MannDude (Jan 12, 2014)

Didn't realize a thread already existed for this. Been testing it out on a dev VPS, nothing on it. Running ServerBear benchmark script on the VPS right now and watching the stats change on the NodeQuery profile for the server.

I agree that I think it'd be better open-source and offered as a self-hosted solution, but so far I enjoy it. Unsure if I would use it yet in a production environment due to the concerns about the security and privacy of remotely hosted software like this, but time will tell.


----------



## MannDude (Jan 13, 2014)

Monitoring a spare AnyNode VPS I had that was used for a dev project in the past. Pretty neat 

EDIT: Ran the Serverbear Benchmarking script, hence the load.


----------



## NodeBytes (Jan 13, 2014)

It's pretty. If it was self hosted I'd be all over it.


----------



## peterw (Jan 13, 2014)

MannDude said:


> I agree that I think it'd be better open-source and offered as a self-hosted solution, but so far I enjoy it. Unsure if I would use it yet in a production environment due to the concerns about the security and privacy of remotely hosted software like this, but time will tell.


It is a nice project but I will never run a unknown code with root access.


----------



## NQ-Joe (Jan 13, 2014)

Seeing that many of you are less concerned about the hosted part but the fact that the agent is running in root is quite the relief. As I already wrote, none of the script functions actually need root so I will definitely try to change that.

Releasing the backends source is currently not an option for me. I will however think about possible compromises for the future.

I still would like to have more users testing it so if you know anyone who might be interested, just spread the word. A recommendation from a trusted community member or friend is far more worth than what I could achieve with ads or app indexes.

Thank you everyone for your help and feedback, keep it coming


----------



## blergh (Jan 13, 2014)

NQ-Joe said:


> Seeing that many of you are less concerned about the hosted part but the fact that the agent is running in root is quite the relief. As I already wrote, none of the script functions actually need root so I will definitely try to change that.
> 
> 
> Releasing the backends source is currently not an option for me. I will however think about possible compromises for the future.
> ...


If you could write us a nice agreement/TOS & sort the privs out (which should be quick) then i think it would work out better for you with it being SaaS. Unfortunately it doesn't really add up as of now.


----------



## NQ-Joe (Jan 13, 2014)

blergh said:


> If you could write us a nice agreement/TOS & sort the privs out (which should be quick) then i think it would work out better for you with it being SaaS. Unfortunately it doesn't really add up as of now.


Of course, did you spot any specific paragraphs in the current agreement you dislike or would like to have more detailed?


----------



## Melon (Jan 13, 2014)

Also, just noticed the 10-server limit. Not enough for a LEB-hoarder. Does it _really_ use that much of your resources monitoring 10 boxes, or is this just a way of preparing to make money from the project? Even Uptime Robot which has no intention of going paid allows 50.


----------



## Aldryic C'boas (Jan 13, 2014)

Maybe he's being careful, and getting a feel of how resources will be used rather than biting off more than he can chew.


----------



## NQ-Joe (Jan 13, 2014)

Melon said:


> Also, just noticed the 10-server limit. Not enough for a LEB-hoarder. Does it _really_ use that much of your resources monitoring 10 boxes, or is this just a way of preparing to make money from the project? Even Uptime Robot which has no intention of going paid allows 50.


For now that is just the initial beta limit. One of the reasons for the public beta is to determine how much storage the load data will consume so I get an idea about data retention. Feel free to send an email from the address you signed up with and I'll up your limit.



Aldryic C said:


> Maybe he's being careful, and getting a feel of how resources will be used rather than biting off more than he can chew.


Right to the point, exactly


----------



## k0nsl (Jan 13, 2014)

I promoted your service on Twitter, and added one box to test.


----------



## NQ-Joe (Jan 13, 2014)

k0nsl said:


> I promoted your service on Twitter, and added one box to test.


Awesome, thank you very much. Let me know if you have any questions or suggestions.


----------



## NickM (Feb 13, 2014)

Just signed up because it looks interesting.  I haven't really had a chance to give it a spin, but I noticed one minor thing when logging in that I just had to mention.  When you press TAB after entering your email address, the focus should go directly to the password field.  Right now, it goes to the "Forgot" link.  This can be solved by adding tabindex="1" to the email field, and tabindex="2" to the password field.  I know, it's a minor nitpick, but it can really get in the way if you're not expecting it.

Besides that, everything that I've seen so far looks pretty good.


----------



## NQ-Joe (Feb 13, 2014)

NickM said:


> Just signed up because it looks interesting.  I haven't really had a chance to give it a spin, but I noticed one minor thing when logging in that I just had to mention.  When you press TAB after entering your email address, the focus should go directly to the password field.  Right now, it goes to the "Forgot" link.  This can be solved by adding tabindex="1" to the email field, and tabindex="2" to the password field.  I know, it's a minor nitpick, but it can really get in the way if you're not expecting it.
> 
> Besides that, everything that I've seen so far looks pretty good.


Thank you very much Nick. I am using Firefox which does not seem to have that glitch. I will fix it with the next web app update. I already have a few smaller errors on my list but don't want to commit an update just yet.


----------



## HenriqueSousa - WebUp 24/7 (Feb 13, 2014)

I signed up for this service and it is very good.

I would like to suggest you the option to add more warning email addresses, for example I wan my System Administrator to receive the warnings.

keep up the good work 

- Henrique


----------



## NodeWest-Dan (Feb 13, 2014)

I like this.  I'd like it if i could get SMS alerts and email alerts to multiple users.


----------



## wdq (Feb 13, 2014)

I really like this. It combines the uptime monitoring tools with the resource monitoring tools.

A self hosted option would be a great addition in the future, even if it's something that is licensed with a fee.


----------



## NQ-Joe (Feb 14, 2014)

HenriqueSousa - WebUp 24/7 said:


> I signed up for this service and it is very good.
> 
> I would like to suggest you the option to add more warning email addresses, for example I wan my System Administrator to receive the warnings.
> 
> ...


I am currently working on some changes to the notification system, using multiple email addresses is definitely something to implement. SMS alerts will most likely only be available when a pricing model is introduced.


----------



## tortau (Feb 14, 2014)

Figured I should throw in some mix to the discussion. Personally, I don't think @NQ-Joe should release backend code and/or make things open-source (other than the client) because that would just undermine the purpose of his hosted service. It's kinda like asking UptimeRobot to give us their backend code so that we can run the same exact thing in our own environment and not pay for the development time.

Having said that, an API is always appreciated (I know it's in the books, but figured a +1 would be appreciated) so that we can incorporate the aggregate stats/charts on websites. And/or a way to make the stats public (re: public static page).


----------



## NQ-Joe (Feb 15, 2014)

Thank you @tortau. A self-hosted version would also introduce much more support work and problems because of all the different system stacks out there. The idea of a simple monitoring service just works better with a central web service.

The API is already on the list and we will update the progress on Twitter over the next few weeks.


----------



## dave (Feb 15, 2014)

Any idea how much the < 10 server version will cost after the beta period?


----------



## NQ-Joe (Feb 15, 2014)

dave said:


> Any idea how much the < 10 server version will cost after the beta period?


Not really, the public beta started just a few weeks ago and we need more time to estimate things. I am hoping it to be in the price range of a low end VPS though.


----------



## NodeWest-Dan (Feb 15, 2014)

Any need for any team members?


----------



## NQ-Joe (Feb 16, 2014)

NodeWest-Dan said:


> Any need for any team members?


Thank you Dan, not right now but if things go well we could definitely need help in the future. Follow us on Twitter if you want to be updated about those things.


----------



## GIANT_CRAB (Mar 1, 2014)

Hi, I am unable to register.

It just gives me the error "Session expired".

Web browser is Opera 12 on Windows 7.


----------



## NQ-Joe (Mar 1, 2014)

GIANT_CRAB said:


> Hi, I am unable to register.
> 
> It just gives me the error "Session expired".
> 
> Web browser is Opera 12 on Windows 7.


Do you allow session cookies to be sent? Our forms use a CSRF token to validate the integrity of a request. "Session expired" is basically the error whenever something is wrong with those tokens.


----------



## GIANT_CRAB (Mar 1, 2014)

NQ-Joe said:


> Do you allow session cookies to be sent? Our forms use a CSRF token to validate the integrity of a request. "Session expired" is basically the error whenever something is wrong with those tokens.


Yes, but I don't see any sessions or cookies stored on my browser.


----------



## NQ-Joe (Mar 7, 2014)

GIANT_CRAB said:


> Yes, but I don't see any sessions or cookies stored on my browser.


I am sorry, I totally missed your last reply. Did you by any chance try another browser? I've tested with Opera on Mac and it was working fine.


----------



## clarity (Mar 7, 2014)

I have to say that I created an account here today, and the monitoring is pretty awesome! For me the cost after beta will be pretty important, but I am enjoying it right now. I just got my first email for a load issue, and the interface is very nice. A simple, clean design that shows you everything that I would want to know.


----------



## GIANT_CRAB (Mar 8, 2014)

NQ-Joe said:


> I am sorry, I totally missed your last reply. Did you by any chance try another browser? I've tested with Opera on Mac and it was working fine.


Works fine on other browsers, just not this Opera 12.

I think you might using the web kit Opera (Opera 13+) to test.

Really odd.


----------



## BeastVM_Aaron (Mar 8, 2014)

Is nodequery like ''nodewatch'' ? Looks better than ''nodewatch'' so if NodeQuery got same features or more i'd like to try NodeQuery then 

Is NodeQuery got web based control panel ?


----------



## NQ-Joe (Mar 9, 2014)

GIANT_CRAB said:


> Works fine on other browsers, just not this Opera 12.
> 
> I think you might using the web kit Opera (Opera 13+) to test.
> 
> Really odd.


I checked again with Opera 12.0.1 on Mac and the cookies were working fine. However, the graphs were not so I wouldn't recommend using it with our service anyway.



BeastVM_Aaron said:


> Is nodequery like ''nodewatch'' ? Looks better than ''nodewatch'' so if NodeQuery got same features or more i'd like to try NodeQuery then
> 
> Is NodeQuery got web based control panel ?


I didn't know Nodewatch before but what I understood from their website is that they provide anti-abuse monitoring. Our service is much simpler, providing you with basic metrics like uptime, system and network load. It is not a replacement for enterprise monitoring but rather a simple solution for developers and small servers.


----------



## GIANT_CRAB (Mar 9, 2014)

NQ-Joe said:


> I checked again with Opera 12.0.1 on Mac and the cookies were working fine. However, the graphs were not so I wouldn't recommend using it with our service anyway.
> 
> I didn't know Nodewatch before but what I understood from their website is that they provide anti-abuse monitoring. Our service is much simpler, providing you with basic metrics like uptime, system and network load. It is not a replacement for enterprise monitoring but rather a simple solution for developers and small servers.


....

I've just registered using Firefox instead =_=

EDIT:

First bug I noticed - my disk is a 1TB drive yet they displayed 2TB total.

Used 26GB yet it displayed 68GB.

(Btw, I'm using SSD caching via Flashcache (writeback) and the SSD is 128GB with 100GB partition used for cache.)


----------



## NQ-Joe (Mar 10, 2014)

GIANT_CRAB said:


> ....
> 
> I've just registered using Firefox instead =_=
> 
> ...


Thank you for your feedback. Detecting the correct disk space is a little tricky at times. Would you mind sending me the output of 'df' via email or private message?


----------



## dano (Mar 25, 2014)

Finally had a moment to try out NodeQuery -- so far, it's quite simple and has a nice interface/frontend that is fast, and displays useful data that is important to most people that run systems. From what I can tell so far, the single node I have configured seems to be sending accurate data.

Thanks for the public beta trial, and I will let you know if I see anything odd or behaving strange - good job NQ-joe!


----------

