amuck-landowner

Big RAM VPS - who actually uses them and for what?

drmike

100% Tier-1 Gogent
It has been a while since I was last roped into trying one of the abundant 2GB and larger VPS plans.

Last go round (and I tried multiple providers over six month span) all failed.  Usually the provider would have a fit as I loaded a test dataset into MySQL and consumed disk IO + CPU.

While these plans might play with the face value of large RAM, they are often crushed by hard set limits on IOPs, CPU utilization and other "good behavior" limiters.

So, to those of you buying and using these plans, what have you managed to get working and what success have you had running anything over say at least a month.   Looking at use scenarios and no provider names (and if you are a provider, no talking about your own special container without the limitations).
 

mikho

Not to be taken seriously, ever!
I think the vps with most RAM i have has 512mb with 1024 burst.


Currently down and has been for some time, only because I've been to lazy to submit a ticket about it.
 

sundaymouse

New Member
Most hard set limits are on OpenVZ, where it is possible for providers to sell XXL containers at low-end price grades. It really depends on how much CPU and IO power you continuously use.

For example, if you have a 2GB Xen PV VPS (you can get that for about $20/mo somewhere now), with good CPU dispatch policy, if your high CPU+IO load only happens occasionally, I don't think the provider will really mind. At least Linode doesn't mind my occasional ffmpeg encoding (about one hour each week).

But, if you use 1000 IOPS and 2.0 load all the time, probably the only option is to go for a cheap dedi?
 

Alto

New Member
In the past, I tend to have used the higher RAM plans where I'm unsure what my usage is going to be and I don't want to be a bad node neighbour; I normally scrap the plans once I have a decent idea of what I really need.
 

lbft

New Member
While these plans might play with the face value of large RAM, they are often crushed by hard set limits on IOPs, CPU utilization and other "good behavior" limiters.
High RAM oversold-type VPSes (I mean the >=2GB $7 type) are most useful for trading memory for some other resource (disk I/O, CPU usage, etc.)

Cache, cache, cache!

  • Tune your database (e.g. MySQL) so your whole DB fits in memory - that way reads don't need to hit disk at all and you only have to use precious IOPS for writes. It's great for a read-heavy workload, but watch your CPU usage. For a heavy random write workload you're likely better off with an SSD VPS.
    Even better, if you have a largely static dataset, load it into an in-memory table at boot and never hit the disk at all.
  • For file-based stuff stick it in a tmpfs - even on OpenVZ I haven't come across a situation where I couldn't make a great big tmpfs (like, 90% of the VPS's memory size) if I wanted to, although I think there's something in beancounters that can limit its size if a provider so desired.
[*]If your data suits it, cache writes in memory too. It's especially great for data that is replaced frequently (e.g. stats, current statuses, etc. for stuff like characters in games and for monitoring systems). You have to be able to live with losing data not yet pushed to disk in case of power cut/unexpected reboot though, and I don't know of any off-the-shelf web apps that do this.
[*]Cache objects generated from the database in shared memory or memcached (or Redis I suppose, but I have no experience there).
[*]Store user session data in memory.
[*]Cache chunks of generated HTML (MediaWiki does this for its UI, for example, and it's an integral part of reddit's caching strategy where it saves them a bunch of latency and CPU usage despite having highly dynamic pages). CloudFlare's Railgun is a similar idea where they cache chunks of HTML to save transferring it over the network.
[*]Cache entire rendered pages. I know personally that nginx has useful stuff here like FastCGI caching and proxy caching. Stick the cache on a tmpfs. nginx even has a module to serve files directly from memcached. I'm sure other web servers have similar options, or otherwise stick Varnish in front (may require carefulness on OVZ, since last I heard it was a bit wonky when you mmap everything in the entire universe like Varnish and MongoDB do, although I notice that Varnish seems to have a memory-only storage backend these days).
Even then unless you specifically design your app to use a crapton of caching, and have a sufficiently large data set, you're going to bump into other limits before you use all your memory on some of the larger offers. And in that case depending on your data usage pattern you may run out of disk space first. And then there's the problem that cache invalidation is hard.

On the other hand, the biggest advantage of high RAM plans isn't using it at all - it's simply that you can choose to never have to worry about memory usage, because you're never ever going to be able to use it all accidentally.

TL;DR: don't configure your DB like it's a 128, put EVERYTHING in an in-memory data store, and forget about using it all anyway.
 
Last edited by a moderator:

telephone

New Member
I have quite a few high RAM apps in use (only on KVM):

- Gitlab (eats RAM for breakfast!)

- Jenkins

- Plex with Sick-Beard/SABnzbd

- Turn caching up (MySQL, Nginx, and OPcache)

- XFCE desktop via x2go

The only warning I've received was for Plex transcoding, and that was just a slap on the wrist as I had 5 devices transcoding at once  B)
 
Last edited by a moderator:

SrsX

Banned
In my mind, anything over 4GB of ram - I'd rather just get a dedicated server.

The biggest VPS I have is a 3GB ram one, which runs testing and skype. I run Skype on this due to the fact, there are kids out there who will skype resolve you and ddos you offline - so I'd rather they hit my VPS which is behind Voxility firewall - because it just pisses them off the fact they can't, plus it saves me having to call my ISP and ask them to assign me a new static IP.
 

bdtech

New Member
Cache, cache, cache!

  • Cache entire rendered pages. I know personally that nginx has useful stuff here like FastCGI caching and proxy caching. Stick the cache on a tmpfs. nginx even has a module to serve files directly from memcached. I'm sure other web servers have similar options, or otherwise stick Varnish in front (may require carefulness on OVZ, since last I heard it was a bit wonky when you mmap everything in the entire universe like Varnish and MongoDB do, although I notice that Varnish seems to have a memory-only storage backend these days).
Is intentionally tmpfs'ing necessary? If the file is "HIT" enough, and you have free RAM available, Linux should cache it to RAM automatically.
 
Last edited by a moderator:

drmike

100% Tier-1 Gogent
I have quite a few high RAM apps in use (only on KVM):

- Gitlab (eats RAM for breakfast!)

- Jenkins

- Plex with Sick-Beard/SABnzbd

- Turn caching up (MySQL, Nginx, and OPcache)

- XFCE desktop via x2go

The only warning I've received was for Plex transcoding, and that was just a slap on the wrist as I had 5 devices transcoding at once  B)
These are good uses.  You mentioned KVM... Take it these are higher priced plans?  Not the < $7 month ones?
 

drmike

100% Tier-1 Gogent
RAM abuse/use ---assuming here that you actually have RAM to use in such ways and the server isn't smacking SSDs mocking RAM....

Only way it really would work is if you have other servers in the same facility with super low latency....

I mean it will work if your other servers are 50ms away, but defeats most case/uses of such (performance).
 

Darwin

Member
Cache, cache, cache!

TL;DR: don't configure your DB like it's a 128, put EVERYTHING in an in-memory data store, and forget about using it all anyway.

This. Cache cache and cache. If you know what you are doing you can develop an app that scales a lot because of cache usage. If you don't develop webapps and use something already baked, you can always use varnish and use a lot of ram to serve 10000s of requests :)
 

DomainBop

Dormant VPSB Pathogen
I bought a 4GB Xen VPS last week.  Primary reason for buying: 500GB of backup space in a location that's <1ms from my SeFlow dedicated servers and <0.5 from my IWStack VPS.

I also have a 4GB RAM  VPS in Iceland (purchased for location and the 500Gb of disk space), and a few  >2GB VPS's in the Netherlands (used for smaller sites, development) but they're all on E5's and none of them are used for high traffic sites or CPU/IO intensive applications.  The heavy duty stuff all goes on dedicateds.  I won't buy any big RAM plan on an E3 because the performance and stability of the ones I've tried has...sucked.
 

nunim

VPS Junkie
The only VPS that I have with more than 512MB of ram runs Windows. Even then I could probably get by with 1GB as I just use it for RDP.


I'm a big fan of optimization, I've got Observium running on a 128 RamNode SSD monitoring 35 servers and hosting a frequently used looking glass, still hardly runs above .2 load with 30MB or so to spare.

observium_ram_12_31_13.png
 
Last edited by a moderator:

Tux

DigitialOcean? lel
I use big RAM VPSes due to the sheer amount of Java programs running on my machines, though I only really only have one 1GB VPS with RamNode doing that. I'm looking into upgrading that to 2GB for a project.
 
Last edited by a moderator:

tchen

New Member
I'm running a small logstash setup on one of the 3GB RAM series.  There's a Redis on the front to queue incoming syslog messages, with the regex'ing done by another logstash (yes, running two) - dumping it all into elasticsearch.  I haven't heard a peep from the provider so I assume it's not much of a load CPU/IO wise.
 

telephone

New Member
These are good uses.  You mentioned KVM... Take it these are higher priced plans?  Not the < $7 month ones?

Nope, you just have to jump on the specials  ;)

I've had a myriad of them ranging from 2 GB to 4 GB, and all under $10 a month.

^ I'm down to just one now.
 
Last edited by a moderator:

lbft

New Member
Is intentionally tmpfs'ing necessary? If the file is "HIT" enough, and you have free RAM available, Linux should catch it to RAM automatically.
It's almost certainly not necessary but I'm paranoid about disk I/O usage on VPSes (I don't want to be a noisy neighbour), especially OpenVZ where I can't monitor how many IOPS I'm using with iostat.
 

wlanboy

Content Contributer
Most of my vps do have 128 MB of RAM.

Enough space for about everything.

Webstack (even VestaCP), mail servers, Ruby workers, MongoDB arbiters, RabbitMQ cluster nodes, etc.

But I do have three high RAM vps (all from specials):

  • 1 GB OpenVZ - my site's backbone - (Redis + MongoDB master + RabbitMQ master + a lot of crons)
  • 2 GB KVM - my Java world - (Jetty + Hudson + Ivy + Junit + Ant builder)
  • 2 GB KVM - my devbox - (Git + a lot of LXC containers as staging environments)
Regarding the last one:

I used my backups to create 1:1 copies of my vps as lxc containers.

They cannot run all at the same time but if I want to test upgrades or new versions of software it is quite easy to do:

  • start lxc container
  • git pull + schema update
  • run tests
  • feel good and update the real vps
Saved my ass on some migrations.
 

mikho

Not to be taken seriously, ever!
I think the vps with most RAM i have has 512mb with 1024 burst.


Currently down and has been for some time, only because I've been to lazy to submit a ticket about it.
Forgot all about it until Windows was mentioned. i do have a vmware vm with 3gb RAM @ $5/month for my online stuff.
 

willie

Active Member
I've been involved (at work) in using large EC2 instances to host database servers.  They work ok for that, but are a LOT more expensive than the supposed 2GB $7/month plans we see on this board and LEB.

I have a 1GB OpenVZ plan with ipxcore but I got it mostly for its largish disk storage.  I can't think of any times I actually did anything that used much of the ram.  I guess I could imagine running Redis in a vps like that, and see whether it got swapped. 

Most non-work things I do that want a lot of ram, also want enough cpu to drive shared vps hosts into conniptions.  So I do that stuff on dedicated servers.
 
Top
amuck-landowner