Running haveged to enhance your system entropy

[wlanboy]

Encryption is based on two main factors:

• prime numbers
• random numbers

On Linux the random numbers are generated by the pseudo random number generator.

It generates randomness from hardware interrupts e.g. by keyboard, mouse, disk or network I/O.

The main difference between /dev/random and /dev/urandom is that first is a blocking device.

So /dev/random waits until the entropy pool is filled to return random data.

/dev/urandom does not wait and therefore gerenates random data in a lower quality.

Lower quality means that previous data is repeated more likely.

So the quality of the entropy pool does have a direct inpact on the quality of SSL/TLS and other block ciphers.

One of my small servers does have a quite low entropy level:


cat /proc/sys/kernel/random/entropy_avail
129

One of my big (and busy) KVM servers does not have that problem:


cat /proc/sys/kernel/random/entropy_avail
4968

A level beyond 1000 is critical if you are using any OpenSSL based encryption.

If your server is handling a lot of SSL traffic (and handshakes) like any webserver or mailserver does, two things can happen:

• The server waits until /dev/random is ready again
• The server switches to /dev/urandom or it's own random generator (after a timeout)

There are userland solution like the audio_entropyd or the video_entropyd to feed the entropy pool, but they are not usable for headless servers.

But there are solutions for servers too - one of them is haveged.

It is a daemon that feeds the /dev/random pool on Linux using an adaptation of the HArdware Volatile Entropy Gathering and Expansion algorithm.

Installation is simple:


apt-get install haveged

Afterwards you have to check the default settings to ensure that enough bits are generated:


nano /etc/default/haveged

Content:


# Configuration file for haveged

# Options to pass to haveged:
# -w sets low entropy watermark (in bits)
DAEMON_ARGS="-w 2048"

After starting the server:


service haveged start

You should then set the daemon to autostart:


update-rc.d haveged defaults

There are - of course - better tests than cat to check if your entropy is ok:


apt-get install rng-tools

rng-test is using the FIPS-140 method to check the entropy:


cat /dev/random | rngtest -c 1000

Output would be:


rngtest 2-unofficial-mt.14
Copyright (c) 2004 by Henrique de Moraes Holschuh
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

rngtest: starting FIPS tests...
rngtest: bits received from input: 2000032
rngtest: FIPS 140-2 successes: 1000
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=8.557; avg=17.635; max=24.236)Mibits/s
rngtest: FIPS tests speed: (min=68.610; avg=156.648; max=188.846)Mibits/s
rngtest: Program run time: 133849 microseconds

On 1000 runs there should not be more than 1 to 5 failures.

So check the line:

rngtest: FIPS 140-2 failures:

 

[tonyg]

Are there any benefits to using haveged over rng?

 

[lbft]

If the root user of any OpenVZ VPS can add entropy to the pool, would it be possible for one VPS to feed the pool a malicious entropy source and affect the security of other VPSes on the same node?

 

[tchen]

If the root user of any OpenVZ VPS can add entropy to the pool, would it be possible for one VPS to feed the pool a malicious entropy source and affect the security of other VPSes on the same node?

You shouldn't be able to feed /dev/random from within a container.  Haveged, TimerEntropy nor any other generator won't run in a guest VE.  Well, they'd run, but won't be able to set anything

 

[lbft]

You shouldn't be able to feed /dev/random from within a container.  Haveged, TimerEntropy nor any other generator won't run in a guest VE.  Well, they'd run, but won't be able to set anything

I had assumed that, but @wlanboy's post seems to imply that haveged does in fact run and put entropy into the pool.

 

[tchen]

I had assumed that, but @wlanboy's post seems to imply that haveged does in fact run and put entropy into the pool.

It does.  But only within KVM and other full virtualization environments.   (sorry, VE in the previous context was OpenVZ VE) 

edit: oh, I get what you mean now.  Ya, he left it unsaid. 

 

[lbft]

It does.  But only within KVM and other full virtualization environments.   (sorry, VE in the previous context was OpenVZ VE) 

I understand what you mean, but I'm suggesting that the way I read the post, wlanboy was suggesting that it works inside an OVZ container:

One of my small OpenVZ servers does have a quite low entropy level:




cat /proc/sys/kernel/random/entropy_avail
129

One of my big (and busy) KVM servers does not have that problem:




cat /proc/sys/kernel/random/entropy_avail
4968

[...]

After starting the server [...] the entropy level is raising:




cat /proc/sys/kernel/random/entropy_avail
142
cat /proc/sys/kernel/random/entropy_avail
153
cat /proc/sys/kernel/random/entropy_avail
170
cat /proc/sys/kernel/random/entropy_avail
188

[...]

It wasn't a suprise that KVM beats OpenVZ if it comes to entropy, but at least you can do something to solve this - even if your providers does not want to help.

The entropy figures quoted after starting haveged only make sense compared to the initial OpenVZ numbers, and the conclusion only makes sense if you take the "something to solve" as referring to low available entropy inside an OpenVZ container, given his conclusion that the KVM VPS had gathered adequate entropy in the first place. The reference to providers not helping strongly suggests this is not to do with increasing entropy for the operator of an OpenVZ host node, either.

 

[KuJoe]

We use rngd for our servers but I decided to give haveged a try and ran some tests, here's the results:

OpenVZ Node with rngd:


# cat /proc/sys/kernel/random/entropy_avail
3968
 

]# cat /dev/random | rngtest -c 1000
rngtest 2
Copyright (c) 2004 by Henrique de Moraes Holschuh
This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
rngtest: starting FIPS tests...
rngtest: bits received from input: 20000032
rngtest: FIPS 140-2 successes: 1000
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=10.007; avg=15.109; max=19073.486)Mibits/s
rngtest: FIPS tests speed: (min=89.969; avg=108.405; max=111.541)Mibits/s
rngtest: Program run time: 1438734 microseconds


OpenVZ Node with haveged:


# cat /proc/sys/kernel/random/entropy_avail
1147
 
# cat /dev/random | rngtest -c 1000
rngtest 2
Copyright (c) 2004 by Henrique de Moraes Holschuh
This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
rngtest: starting FIPS tests...
rngtest: bits received from input: 20000032
rngtest: FIPS 140-2 successes: 1000
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=1.141; avg=22.261; max=30.132)Mibits/s
rngtest: FIPS tests speed: (min=79.143; avg=106.370; max=111.541)Mibits/s
rngtest: Program run time: 1036496 microseconds

OpenVZ Node with both:


# cat /proc/sys/kernel/random/entropy_avail
3968
 

# cat /dev/random | rngtest -c 1000
rngtest 2
Copyright (c) 2004 by Henrique de Moraes Holschuh
This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
rngtest: starting FIPS tests...
rngtest: bits received from input: 20000032
rngtest: FIPS 140-2 successes: 1000
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=3.736; avg=18.521; max=19073.486)Mibits/s
rngtest: FIPS tests speed: (min=85.531; avg=99.397; max=110.892)Mibits/s
rngtest: Program run time: 1222103 microseconds


And compared to the same OpenVZ node without either:

# cat /proc/sys/kernel/random/entropy_avail
154

 

[Shados]

wlanboy may have just mis-interpreted the data there - with haveged, you wouldn't see a slow, slight increase like that, you'd see it dramatically/immediately spike to >~1024 and spike back up there whenever it dropped below it. And yes, haveged will not work in OpenVZ. Also, in general you can't necessarily trust haveged in a virtual environment - it depends on how the VE treats RDTSC calls.

EDIT: @KuJoe you might want to set haveged to refill when it drops below 4000 or so instead of the default 1024 given rngd is keeping the pool at 3968 bits, then re-run the test. Probably won't be much difference though.

 

[wlanboy]

If the root user of any OpenVZ VPS can add entropy to the pool, would it be possible for one VPS to feed the pool a malicious entropy source and affect the security of other VPSes on the same node?

Nope:


echo "10" > /dev/random
bash: /dev/random: Operation not permitted

You shouldn't be able to feed /dev/random from within a container.  Haveged, TimerEntropy nor any other generator won't run in a guest VE.  Well, they'd run, but won't be able to set anything

Yup it is a shared resource.

wlanboy may have just mis-interpreted the data there - with haveged, you wouldn't see a slow, slight increase like that, you'd see it dramatically/immediately spike to >~1024 and spike back up there whenever it dropped below it. And yes, haveged will not work in OpenVZ. Also, in general you can't necessarily trust haveged in a virtual environment - it depends on how the VE treats RDTSC calls.

EDIT: @KuJoe you might want to set haveged to refill when it drops below 4000 or so instead of the default 1024 given rngd is keeping the pool at 3968 bits, then re-run the test. Probably won't be much difference though.

Yup - correct.

Sorry for the confusion - /dev/random is a shared source on OpenVZ.

You can run haveged only on full virtualization like on KVM.

Updated my post.

 

[drmike]

Thanks for this Wlanboy!   I've used both approaches in the past on dedicated servers....

I've grafted this into my installation scripts here forward.  Keep up the great work!

 

[Magiobiwan]

I remember finding haveged a couple months ago. Pushed it to servers and suddenly everything got MUCH faster. Our newest nodes with v3 CPUs and rrand were doing okay before the addition of haveged, but rrand wasn't feeding by default (rng-tools not installed or something). 

 

[Deleted]

IIRC haveged uses AES/AVX instructions on the CPU to seed the random number generator. The old method is slow because it has to use a very expensive ioport() to talk to the ISA bus, which takes a long time.

Also, running this on non ring0 hardware is silly because of privilege instruction emulation, which will probably make things slower.

 

[KuJoe]

I switched off rngd and enabled haveged on our nodes to see if I can notice any difference and I can't say that I do, maybe our clients will though so I'll run it for a month and see what happens.

 

[eva2000]

Interesting thanks for the nice write up

this is what I see on my ramnode 128MB OpenVZ VPS which hosts my SPDY/SSL blog

 

[Nick_A]

Yeah, someone recommended haveged to us a while back. Definitely a nice boost.

 

[D. Strout]

I switched off rngd and enabled haveged on our nodes to see if I can notice any difference and I can't say that I do, maybe our clients will though so I'll run it for a month and see what happens.

My 96MB, mostly idle OpenVZ box with you has 4096 entropy available. A 1024MB OVZ box I have with another provider not running haveged has 149 entropy available. So yeah.

 

[KuJoe]

My 96MB, mostly idle OpenVZ box with you has 4096 entropy available. A 1024MB OVZ box I have with another provider not running haveged has 149 entropy available. So yeah.

Before we switched to haveged it was 3986 entropy. I'm keeping rngd in our build template for now but I might change it to haveged depending on if I notice any difference or we get any tickets as a result over the next month. 

 

[eva2000]

Yeah, someone recommended haveged to us a while back. Definitely a nice boost.

my blog thanks you for it

 

[wlanboy]

I remember finding haveged a couple months ago. Pushed it to servers and suddenly everything got MUCH faster. Our newest nodes with v3 CPUs and rrand were doing okay before the addition of haveged, but rrand wasn't feeding by default (rng-tools not installed or something). 

I switched off rngd and enabled haveged on our nodes to see if I can notice any difference and I can't say that I do, maybe our clients will though so I'll run it for a month and see what happens.

Yeah, someone recommended haveged to us a while back. Definitely a nice boost.

Checked this and the vps I have with theses providers do have the highest entropy levels of my vps pool.

This is for the OpenVZ offers - on KVM you have to fill your entropy pool on your own.