amuck-landowner

How To: Determining how many 'VPS neighbors' you have or if you are on an oversold OpenVZ node

DomainBop

Dormant VPSB Pathogen
There are actually a few commands you can throw at a container that can tell you whether or not it's nested inside Xen/KVM, but that can be for a rainy day or something. :)
https://github.com/kaniini/slabbed-or-not / detects slabs nested inside Xen/KVM/VMware/Hyper-V/bhyve

the companies putting 1000 containers on a box are outed, they probably deserved it because it is companies like that that have destroyed OpenVZ's reputation.
Overloaders are a problem but the inexperienced hosts (you know, the ones whose experience consists of hosting a Minecraft server for a couple of friends) who tend to favor OVZ over other methods because they think it is "easier" than Xen/KVM and can be oversold more have probably done more damage to its reputation. 
 
Last edited by a moderator:

Geek

Technolojesus
Verified Provider
I've seen that slabbed or not thing, it just runs the same commands I would anyhow :p Can't argue with CPU features.
 

KuJoe

Well-Known Member
Verified Provider
You get some of these container counts we have have seen 600-1600 on a server and I fail to see how such could perform even at an acceptable level.   Only math at play there is scores of idle containers online, but sitting with no real use - only way such is viable as a loading stunt.  Even those better be running on an E5 to make it believable.
600 32MB containers != 600 4GB containers so again, without any additional information about the containers having a container count on the node is useless. You are correct also about the idle containers, that's why if the number of containers included usage info it would be much better to get an idea from but having the number of containers itself can only lead to speculation and not real facts and we all know how speculation can hurt a provider more than actual facts so assuming the performance will be poor based on a number without anything else to support it will only be bad news for those providers who utilize OpenVZ and do it correctly like @Geek said.
 

devonblzx

New Member
Verified Provider
600 32MB containers != 600 4GB containers so again, without any additional information about the containers having a container count on the node is useless.
I do understand your argument, there does require some context, but I think the context is the responsiveness of your server.  If you see a lot of wait, or steal time, then you see there are 600 containers being run on the same system, you know what the problem is.

Even with 32MB systems 600 containers is most likely highly oversold.    Let's say you have an E3, 32GB system (probably a common config for most of these places).

Sure 600x32MB is ~19GB.  So the memory isn't oversold (they might even try to put 1000 containers in that case) but what about CPU?  What about disks?  What about network?

That is a 4 physical core system with 600 containers sharing it (probably each stated to be able to use 1 CPU).  You could potentially have 600 different containers trying to access the CPU or disk at the same time, especially if you have cronjobs or scheduled tasks kick off at the same time.

It comes back to the same thing as the Xen can't be oversold argument of 2008.  Yes, there is more to overselling than just memory and disk space.
 
Last edited by a moderator:

KuJoe

Well-Known Member
Verified Provider
I do understand your argument, there does require some context, but I think the context is the responsiveness of your server.  If you see a lot of wait, or steal time, then you see there are 600 containers being run on the same system, you know what the problem is.
 That's a bold assumption that might not be correct though which is my point. If clients see 600 containers and their VPS is slow, they would automatically assume it's because of the 600 containers and not something else and most likely cancel their service instead of opening a ticket to get it resolved.

Even with 32MB systems 600 containers is most likely highly oversold.    Let's say you have an E3, 32GB system (probably a common config for most of these places).

Sure 600x32MB is ~19GB.  So the memory isn't oversold (they might even try to put 1000 containers in that case) but what about CPU?  What about disks?  What about network?

That is a 4 physical core system with 600 containers sharing it (probably each stated to be able to use 1 CPU).  You could potentially have 600 different containers trying to access the CPU or disk at the same time, especially if you have cronjobs or scheduled tasks kick off at the same time.

It comes back to the same thing as the Xen can't be oversold argument of 2008.  Yes, there is more to overselling than just memory and disk space.
 Oversold != diminished performance. Having run a production node with over 600 containers before on older hardware than a Intel E3 I can attest that it does not mean the VPSs would be slow be it CPU, disk IO, or network speed. The provider's abilities and knowledge are much more important in terms of performance than how many containers per node so taking the number of containers and applying it to anything out of context is as accurate as throwing a dart at a wall full of possible other issues, sure you might be correct once in a while but you won't be able to confirm it and assuming you're right won't benefit anybody.
 
Last edited by a moderator:
  • Like
Reactions: raj

DomainBop

Dormant VPSB Pathogen
Having run a production node with over 600 containers before on older hardware than a Intel E3 I can attest that it does not mean the VPSs would be slow be it CPU, disk IO, or network speed.
The "Slow" mentioned by @KuJoe and the "Responsiveness" mentioned by @devonblzx  are both very subjective measure and their definition will vary from customer to customer.  There really aren't any objective measures that can definitively define "oversold" (although when nodes run out of disk space and crash as happened to me with a Dutch provider it's a good indication the provider is massively overselling resources), and for some customer's uses slightly degraded performance on a node due to overselling may be acceptable to them.

As a customer I have a simple test: does the VPS do a good job of doing what I need it to do? If YES, I renew.  If NO, I find a new provider.
 

drmike

100% Tier-1 Gogent
I mean it comes back to the run around of smoking customers and banking on perception as many providers wear the virtual face mask while mugging  deceiving advertising [to] customers... 

Only benefit to consumers for packing nodes to the gills is super low price perhaps.  This is where a $20 VPS package in real shops is selling for < $3 a month on the cheap side.  If some hobbyist is getting a "deal" for their leisure-use of such a machine then I guess that mutual arrangement works.  Problem is 99% of VPS companies aren't selling that way or angle.  They are instead talking about high performance, enterprise and all that stuff that is bragging about how bad ass they are... Then the poor customer quickly feels like a dupe and if customer opens their mouth hole, the provider hides behind the VPS face mask using customer not knowing as way to legitimate their sub-par performance.

Same mugging mask is worn in a lot of shops to lure customers with bullshit claims of 24/7 support by certified ninjas who don't sleep and only live to serve the customer's whims.  Again, how does one prove or disprove such?  Through spot checks and perception indicators.   Often companies fail on this and BS about something to justify the failure.  Happens so much that it's sad.

Having bought from MANY VPS companies over the years I can say firsthand that resource contention, overloading, poor controls on other containers, etc. is precisely why I no longer am with a very long list of companies.  Companies I have services with still very well might overload.  Difference is they inherently get it and go beyond the kid with VPS-biz-in-a-box no knowledge.

This s why outside of a handful of proven companies I won't even try FREE VPS / Trial offer / 30 day refund containers from providers.  Not even worth the waste of my time to see the same fail generic deployments and knowledge level with gear that is meh - not so impressive.

Can you super load xxx - xxxx containers on a server?  Yes.  Will performance suck if any percentage of those folks actually use during same period of time, yes.  Is overloading common?  VERY.  

Oversold to me simply is the following:

    1. Disk ratio sold unreasonable vs. physical available pool size.  This is especially concerning where providers allocate or sell LARGE disk offers.  Small disk allocation shops could be worse as out of disk stands to effect, oh hundreds of customers on a single node.  I even created a commandline tool way back to "reserve" disk space your provider allocates.   Probably caused a few out of disk situations, oh well, bad providers need some ethics time to reflect.

    2.  RAM - hitting out of RAM on the server, going to swap on the bare server.  Also other games to boost RAM via SSD and other hacks.

   3.  xxx - xxxx containers but just a single Gbit uplink.   Unsure why damn near everyone thinks single Gbit is acceptable in a highly shared environment.  Imagine a 600 unit apartment building with a single quarter inch water line.  Gbit isn't even acceptable on my LAN.

   4.  Lack of cores - too many shops still running on dual quad cores or those E3's.  Nice idea where cheap gear buy and right dollar cost vs. loading.  Power packing that stuff meh.

   5. Stacking customers dense like it's just fine.   This EVERYONE IS IDLE guarantee eventually isn't true, someone uses.   Some shops shuffle that off to another server, some call it abuse, etc.    CPUs are better these days, but task switching still costs.  Slam 10k processes on most machines and it's just bad.

In other industries there are best practices on such over subscription.  Ratios.   In VPS land it remains the wild west, whatever one can get away with with a light observation of performance at best, often after the customers complain.

Lack of transparency about servers and real world performance is what is allowing kids and know-nothings to spring up and compete toe-to-toe.  The gaming and masking things is what is creating STUPID and ARTIFICIAL competition in the marketplace.  No way to differentiate, then the guy on a 10 year old server is on same footing as company of 500 employees with certifications running latest E5's with big RAM.

I know my comments here won't win any fans, it's a provider dense audience.  I am a consumer of services first and foremost.  Customers matter and sooner more shops realize this and limit the insane loading the better for customers, the industry and real businesses.  

++++ get to making numbers on servers public transparent.   Maybe not container count, but all other server load data.
 
Last edited by a moderator:

KuJoe

Well-Known Member
Verified Provider
I know my comments here won't win any fans, it's a provider dense audience.  I am a consumer of services first and foremost.  Customers matter and sooner more shops realize this and limit the insane loading the better for customers, the industry and real businesses.
How is raising prices for the same exact service better for the clients?
 
Last edited by a moderator:

drmike

100% Tier-1 Gogent
How is raising prices for the same exact service better for the clients?
This just doesn't add up.  Exact services don't exist unless two know-nothing vanilla installs of OpenVZ on identical hardware and network.  Big big big problem with VPS is most are generic in nature and hardly anyone goes and stands out as different, so assumption is they are all same.   True, most shops are generic and kind of similar.  Maybe most shops are power overloading nodes :)  Shame on them if so.

You are inferring that lowering prices is good.  (sure lowering prices is good for more customer buys / less purchase friction / rejection).   The question is why do so many companies have to go lowering prices to basement levels then go lowering the numbers more?   Are we to adopt this and think that 2GB VPS at $1 a month is reasonable because it's mathematically doable on latest hardware with mass oversell?

Lower prices ARE NOT better for consumers.  If this was true every major hosting BIG BUSINESS would have lowered their numbers to compete with lowend priced shops.  Customers would have demanded lower prices. Big hosting shops are barely lowering prices.  They are MUCH higher per month often than what a lowend per YEAR is.  Lots of customers just won't buy cheap because it's business unsound from support, to lack of phone support, to lack of auditing, to lack of any ABOUT US info, to just sketchy fly-by-night feeling most give.

Raising prices having sustainable prices SHOULD guarantee real business things - like actual staffed help desk working in native tongue and at acceptable knowledge level.  It should pay for monitoring and proactive staffing.  It might pay for actual administrator to manage systems.  It should also have relationship to number of containers on a server and resource contention - this point gets foggier as hardware is more capable of masking the load and contention even during use flare ups.

Way I see it is that a server has potential.  Often that potential is a mirror of the company selling such (the humans, their comforts, self worth, etc.).  Yes, there are some bargain lines from diverse companies that are doing it for biz diversity or market testing or lead gen - not many though.   The potential in a server is $XXX-$XXXX of monthly income.

If I run a more formal host with sustainable numbers I might get $10/GB without sales reduction on income.  I can load a 64GB box 1-1 @ 64 1GB containers (ignoring load and mix use case issues) and have $640 income for 64 containers.

In contrast, the price lowering approach will park hundreds of containers in there, upping contention, upping horror pool when breakage happens pushing oh 150-1000 customers in there and often not even hitting that $640 income number, often.

You are inferring that stacking many hundreds of containers is the exact same service for clients as one loading a few dozen.  It isn't.  Conceptually same, sure.

These issues are why more people are moving away from VPS and distrusting in general.   We dealt with similar can-do approaches with shared years ago.  Difference here is VPS is access to shell and more indicators of slowness and abuse.  That's why VPS customers are less clingy to brands and more likely to up and leave in a heartbeat --- plus a subset is just more technically able to do what they need (not stuck in coddled GUI wrapper).

Whole situation reminds me of the concept of dilution.  Think about it in soda pop terms - as we all probably have had a glass -  you order a Coke or Pepsi it arrives at your table flat and tasting barely like it should.  Why?  Often they are diluting the product with more water to lower their commodity cost on the syrup.  A lowly cheap glass of soda that probably costs the establishment  9 cents (well use to a decade ago) and they are skimming as if their commodity price is too high.  In the process of diluting, they are ruining the reputation of the commodity syrup company (Coke or Pepsi) and they are beating the customer.  

Never though in said restaurants have I seen them take the $1.50-2.50 soda price and lower it to 58 cents just to sell more.  I am entirely sure I've never seen such a place drop things to 58 cents then go intentionally diluting product since they have reduced profitability.  

^ --- This is how I feel lots of VPS shops are operating.  Banking on their customers having paid too little to complain or having spent so little on impulse buying that they aren't even using such services.

Non-active / idle customers might be good for node stretching, but they are horrendous customers.  Retention rates are going to be low since they don't need or want the services.  When a company has tons of that, they should revisit their business concept and customer base.
 

KuJoe

Well-Known Member
Verified Provider
This just doesn't add up.  Exact services don't exist unless two know-nothing vanilla installs of OpenVZ on identical hardware and network.  Big big big problem with VPS is most are generic in nature and hardly anyone goes and stands out as different, so assumption is they are all same.   True, most shops are generic and kind of similar.  Maybe most shops are power overloading nodes :)  Shame on them if so.
I'm talking about the same provider offering the same services for different prices like you are suggesting. If CompanyA is selling a service for $2/month but need to sell it for $20/month to avoid having a high number of VPSs per node then you're still getting the same service from CompanyA but for a higher price. Sure you might get more resources but not everybody wants higher resources and they especially don't want to pay a premium for those resources when they only need a fraction of them.

A lot of companies offer lower prices so consumers don't have to spend a fortune on services they want/need. I personally enjoy offering an affordable service that most people can fit into their budget even if it means not collecting a paycheck myself. I'm taking the hit so clients don't have to and in return I ask that our clients and potential clients don't judge us based on a number that means nothing in the real world.
 
Last edited by a moderator:

drmike

100% Tier-1 Gogent
I'm talking about the same provider offering the same services for different prices like you are suggesting. If CompanyA is selling a service for $2/month but need to sell it for $20/month to avoid having a high number of VPSs per node then you're still getting the same service from CompanyA but for a higher price. Sure you might get more resources but not everybody wants higher resources and they especially don't want to pay a premium for those resources when they only need a fraction of them.
Same provider offering at two price points is doable.  One would be dedicated and guaranteed resources and the other would be pot luck shared.  Typically one is KVM for dedicated and other is OpenVZ on the shared.

Ideally more companies get around to right priced dedicated resource offerings - big lack of such.  I like what ServerAxis does with guaranteed IOPs and some of their terms and marketing.  Shared (OpenVZ) is nice, but it's basically the hosting ghetto like always.  So much destruction has been done to OpenVZ's reputation that I do believe we shall see the death of OpenVZ soon :)

A lot of companies offer lower prices so consumers don't have to spend a fortune on services they want/need. I personally enjoy offering an affordable service that most people can fit into their budget even if it means not collecting a paycheck myself. I'm taking the hit so clients don't have to and in return I ask that our clients and potential clients don't judge us based on a number that means nothing in the real world.
Why not just offer free hosting then?  Strange thing to work for free.  That's a hobby brand and nothing wrong with such if so.  Point where money is not having a paycheck but putting in time is rough.   Idealistic or something, who am I to judge.  Probably been there in different ways myself outside of hosting.  Never stuck to that for years on end though.  Man must have paid work, hobbies and free time.   KuJoe has work + free work + a time machine?  You must :)
 

KuJoe

Well-Known Member
Verified Provider
This thread got me thinking about how we can instill confidence in our clients and potential clients without the need for this useless number and I hope I'm on the right track: http://drgn.biz/servers/

For a fun little experiment, try to guess how many VPSs per node using the ranges below and see if you can determine the node population based on actual performance metrics:

More than 200 VPSs (2 nodes)

100-199 VPSs (3 nodes)

50-99 VPSs (4 nodes)

Less than 50 VPSs (3 nodes)

 

Disclosure: Backups are currently running on some of the nodes so it may skew the performance results a bit.

Also take note of the server name: ovz = OpenVZ, kvm = KVM, bkup = OpenVZ
 
Last edited by a moderator:

devonblzx

New Member
Verified Provider
That's a bold assumption that might not be correct though which is my point. If clients see 600 containers and their VPS is slow, they would automatically assume it's because of the 600 containers and not something else and most likely cancel their service instead of opening a ticket to get it resolved.


Oversold != diminished performance.
That's why I said "If you see a lot of wait or steal time".   That is the context.   Wait time represents the time your CPU is waiting on the disk, Steal time represents the time your virtual processor is waiting on the physical processor.

I think the bold assumption is that with 4 physical cores and 600 containers, you're assuming that 596 containers will be idle at any given point in time.  Otherwise, it does in fact equal diminished performance, even if not very noticeable.
 
Last edited by a moderator:

drmike

100% Tier-1 Gogent
This thread got me thinking about how we can instill confidence in our clients and potential clients without the need for this useless number and I hope I'm on the right track: http://drgn.biz/servers/

For a fun little experiment, try to guess how many VPSs per node using the ranges below and see if you can determine the node population based on actual performance metrics:

More than 200 VPSs (2 nodes)

100-199 VPSs (3 nodes)

50-99 VPSs (4 nodes)

Less than 50 VPSs (3 nodes)

 

Disclosure: Backups are currently running on some of the nodes so it may skew the performance results a bit.

Also take note of the server name: ovz = OpenVZ, kvm = KVM, bkup = OpenVZ
Interesting public view.  I give you mass credit for that.   Hoping more folks adopt a similar approach.

Let's see if I can gander some guesses and get egg on my face :)
 

Fusl

New Member
Well there are two reasons why I am curious:

1) Process count. After about 50000 processes on the host node it will start locking up. If each VPS runs 3 processes (absolute minimum, basically just init and two kernel processes) that's already almost 60000.

2) Even a minimal centos install times 19273 equals 10 TB of data

:)
1) OpenVZ runs fine with (currently counting) 638776 processes if correctly tweaked. As partial OpenVZ developer I pretty much know how to get the most out of it :) (*edit* Also, load average with this amount of processes is at 0.17)

2) Ever heard about deduplication? Compression? Also, we do have that much storage at home, 10TB would not be an issue at all.
 
Last edited by a moderator:

QHoster.com

New Member
What is the meaning of VPS and guaranteed resources then if you place 500 VPSes x 512MB RAM on a 32GB RAM node ?

What is the difference between offering then 512MB RAM or 4GB RAM or 8GB RAM VPS when placing 500 VPSes or number which can cause at any time 150GB RAM usage on a 32GB VPS node ?

"Who will offer more 'resources' for less $ ?" :)
 

zionvps

Member
Verified Provider
Arguments aside, i ran this in one of the OpenVZ containers and i got this

 cat /proc/cgroups
#subsys_name    hierarchy       num_cgroups     enabled
cpuset  3       1       1
cpu     3       1       1
cpuacct 3       1       1
devices 4       1       1
freezer 4       1       1
net_cls 0       1       1
blkio   1       1       1
perf_event      0       1       1
net_prio        0       1       1
memory  2       1       1

My node definitely does not have one container.

Also, your other script's output is 

/# wget http://cdn.content-network.net/Mun/apps/container_counter/script.txt -O - | php
--2015-08-29 15:26:10--  http://cdn.content-network.net/Mun/apps/container_counter/script.txt
Resolving cdn.content-network.net (cdn.content-network.net)... 2607:5600:5cf::2, 192.254.77.173
Connecting to cdn.content-network.net (cdn.content-network.net)|2607:5600:5cf::2|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2712 (2.6K) [text/plain]
Saving to: `STDOUT'

100%[==============================================================================================================================>] 2,712       --.-K/s   in 0s

2015-08-29 15:26:10 (283 MB/s) - written to stdout [2712/2712]


############################################################################
                                Container Counter
############################################################################
    By: Mun
    Ver: 1.0
    Site: https://www.qwdsa.com/converse/threads/container-counter.131/
############################################################################
    CPU(s):
----------------------------------------------------------------------------
 AMD xxxx
 AMD xxxx
 AMD xxxx
 AMD xxxx
----------------------------------------------------------------------------
    Kernel:
----------------------------------------------------------------------------
2.6.32-042stab106.4

----------------------------------------------------------------------------
    Container(s) On Node:
----------------------------------------------------------------------------
0
----------------------------------------------------------------------------
 

Geek

Technolojesus
Verified Provider
Eh, this was patched out and replaced with a fake output a few months ago.

There are other ways of getting some details about the node if you know where to look.  Hopefully mustard fuck can hold it together if I ever decide to post how to determine if your VPS is running on a blade. Being that it's completely irrelevant, there's really no need...
 
Top
amuck-landowner