amuck-landowner

Small redis key/value database

wlanboy

wlanboy

Content Contributer
Redis is a lightweight and fast key/value store. I did have a view projects where I needed something like a persistent hashmap with quite a view values. Too much for a text file but at the same time too simple for MySQL. So Redis fills the gap with a small memory footprint key/value database. And it is fast too :D

Installation is quite simple:


mkdir ~/redis && cd ~/redis
wget http://redis.googlecode.com/files/redis-2.6.13.tar.gz
tar -zxf redis-2.6.13.tar.gz
cd redis-2.6.13
make
sudo make install

Redis is now able to run.

All needed to start the server is calling this command:


cd ~/redis-2.6.13/src && ./redis-server ~/redis.conf

You can also use the command line interface:


cd ~/redis-2.6.13/src && ./redis-cli

But you should use upstart to start/stop the server.

Some preparations:


sudo su
mkdir -p /var/lib/redis
mkdir -p /var/log/redis
useradd --system --home-dir /var/lib/redis redis
chown redis.redis /var/lib/redis
chown redis.redis /var/log/redis

Creating a minimal redis config file (change the password!):


nano /etc/redis.conf

With content:


# Redis configuration file
daemonize yes
pidfile /var/run/redis.pid
port 6379
bind 127.0.0.1
timeout 0

# debug (a lot of information, useful for development/testing)
# verbose (many rarely useful info, but not a mess like the debug level)
# notice (moderately verbose, what you want in production probably)
# warning (only very important / critical messages are logged)
loglevel notice
logfile /var/log/redis/redis.log

# Set the number of databases. The default database is DB 0, you can select
# a different one on a per-connection basis using SELECT <dbid> where
# dbid is a number between 0 and 'databases'-1
databases 5

save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb

# The working directory.
dir /var/lib/redis

slave-serve-stale-data yes
slave-read-only yes
slave-priority 100

################################## SECURITY ###################################
# Warning: since Redis is pretty fast an outside user can try up to
# 150k passwords per second against a good box. This means that you should
# use a very strong password otherwise it will be very easy to break.
#
requirepass [your super strong password]

################################### LIMITS ####################################
maxclients 100
maxmemory 50mb

# volatile-lru -> remove the key with an expire set using an LRU algorithm
# allkeys-lru -> remove any key accordingly to the LRU algorithm
# volatile-random -> remove a random key with an expire set
# allkeys-random -> remove a random key, any key
# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
# noeviction -> don't expire at all, just return an error on write operations
maxmemory-policy volatile-lru
appendonly no
# appendfsync always
appendfsync everysec
# appendfsync no

no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128

hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
activerehashing yes

client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60

And the init file:


nano /etc/init.d/redis-server && chmod +x /etc/init.d/redis-server

With content:


#! /bin/sh
### BEGIN INIT INFO
# Provides: redis-server
# Required-Start: $syslog
# Required-Stop: $syslog
# Should-Start: $local_fs
# Should-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: redis-server
# Description: redis-server
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/redis-server
DAEMON_ARGS=/etc/redis.conf
NAME=redis-server
DESC=redis-server
PIDFILE=/var/run/redis.pid

test -x $DAEMON || exit 0
test -x $DAEMONBOOTSTRAP || exit 0
set -e

case "$1" in
start)
echo -n "Starting $DESC: "
touch $PIDFILE
chown redis:redis $PIDFILE
if start-stop-daemon --start --quiet --umask 007 --pidfile $PIDFILE --chuid redis:redis --exec $DAEMON --background -- $DAEMON_ARGS
then
echo "$NAME."
else
echo "failed"
fi
;;
stop)
echo -n "Stopping $DESC: "
if start-stop-daemon --stop --retry 10 --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
then
echo "$NAME."
else
echo "failed"
fi
rm -f $PIDFILE
;;

restart|force-reload)
${0} stop
${0} start
;;
*)
echo "Usage: /etc/init.d/$NAME {start|stop|restart|force-reload}" >&2
exit 1
;;
esac

exit 0

Autostart of the redis server:


update-rc.d redis-server defaults

Enjoy your own hosted redis database server.

And please keep in mind that redis is not build for safety. Running it as a public service is not a good idea. Just read the comments of the configfile regarding the password. To double check this is the paragraph:


# Warning: since Redis is pretty fast an outside user can try up to
# 150k passwords per second against a good box. This means that you should
# use a very strong password otherwise it will be very easy to break.

.
 
Last edited by a moderator:
drmike

drmike

100% Tier-1 Gogent
Awesome!   Now I need to figure out API/tie in to Redis for some of my programming and give it a spin.
 
peterw

peterw

New Member
Thank you for your post! A lot of ruby programmers are using redis and writing about it but I am working with c++ and php. I want to ask two questions: 1. What are you using redis for? 2. Do you have php examples?
 
wlanboy

wlanboy

Content Contributer
First questions is easy:

Persistent key/value or hash store.

Second question:

For php use predis.

As requested some of my php code:


1. Parameters
$redis->set("param_logging", "on");
$value = $redis->get("param_logging");
echo ($redis->exists("use_proxy")) ? "true" : "false";

2. Counters
$redis->incr("couter_for_something");
$redis->decrby("another_counter", 5);

3. Hashset
$redis->hset("user1", "name", "wlanboy");
$redis->hset("user1", "groupid", "4");
echo "groupid: " . $redis->hget("user1", "groupid");
$redis->hincrby("user1", "number_of_logins", 1);

4. Time to live
$redis->set("todo1", "I live for an hour");
$redis->expire("todo1", 3600);

Second source for ideas is the command reference.
 
trewq

trewq

Active Member
Verified Provider
I have never understand the appeal of redis over mysql. Could someone please help me out and explain?
 
Last edited by a moderator:
V

vampireJ

New Member
trewq said:
I have never understand the appeal of redis over mysql. Could someone please help me out and explain?
Click to expand...
Looks like redis is like memcached- which theoretically can speed up data retrieval. Usually you use redis / memcached along with a database.
 
wlanboy

wlanboy

Content Contributer
vampireJ said:
Looks like redis is like memcached- which theoretically can speed up data retrieval. Usually you use redis / memcached along with a database.
Click to expand...
Yup - memcache is in the same familiy of database systems like redis.

trewq said:
I have never understand the appeal of redis over mysql. Could someone please help me out and explain?
Click to expand...
So lets compare Redis and MySQL:

  • Redis is an in-memory database. Your data has to fit in your free memory.
    MySQL stores the data on disks, and cache part of the data in memory.
  • Redis is a data storage server. There is no query language (SQL) and no support for relations.
  • Redis only offers basic security at the instance level.
    MySQL provides fine grained per-object access control lists.
  • One Redis instance is not scalable. It only runs on one CPU core in single-threaded mode.
    To get scalability, several Redis instances must be deployed and started.


    But the event driven design allows Redis to handle more requests than MySQL. 100K+ operations a second are quite easy to accomplish on a small vps.
  • When it comes to data types Redis supports simple key-value storage just as memcache but it also adds support for Lists.
So why should soemone use Redis:

It is a database which is strip down to some use cases to do them damn fast.

Use cases would be:

  • Session handling
  • Counters
  • Hashmaps (key-value store, great for caches)
  • Lists (ordered collection, great for queue)
  • Sets (unordered collection of non-repeating values)
  • Sorted Sets (ranked collection of non-repeating values, great for Leaderboards)
 
trewq

trewq

Active Member
Verified Provider
wlanboy said:
So why should soemone use Redis:

It is a database which is strip down to some use cases to do them damn fast.

Use cases would be:

  • Session handling
  • Counters
  • Hashmaps (key-value store, great for caches)
  • Lists (ordered collection, great for queue)
  • Sets (unordered collection of non-repeating values)
  • Sorted Sets (ranked collection of non-repeating values, great for Leaderboards)
Click to expand...
Thank you! That is exactly what I wanted to know!
 
wlanboy

wlanboy

Content Contributer
trewq said:
Thank you! That is exactly what I wanted to know!
Click to expand...
Some stories about Redis used by "small" websites:

Digg just rolled out a new feature, cummulative page event counters (page views plus clicks),


that is using Redis as its underlying solution.

Clickstream information is extracted real time from logs and then Redis’s support for


incrementing values comes into play. And in case you are wondering how these counters


deal with concurrent updates, keep in mind that Redis is a single threaded engine,


so all operations are executed sequentially.

In Digg’s own words: “Redis rocks!”
Click to expand...
Using Redis as a Secondary Index for MySQL

Looking back at our original problem, the bottleneck was not in generating the list of photos


for your most recently active contact, it was just in finding who your most recently active contact


was (specifically if you have thousands or tens of thousands of contacts). What if, instead of


fully denormalizing, we just maintain a list of your recently active contacts?


That would allow us to optimize the slow query, much like a native MySQL index would;


instead of needing to look through a list of 20,000 contacts to see which one has uploaded


a photo recently, we only need to look at your most recent 5 or 10 (regardless total count)!
Click to expand...
 
wlanboy

wlanboy

Content Contributer
Taronyu said:
I still prefer the redis php module from dotdeb. :) Works so much easier.
Click to expand...
And it is indeed easier:

Code: 
nano /etc/apt/sources.list.d/dotdeb.org.list
deb http://packages.dotdeb.org squeeze all
deb-src http://packages.dotdeb.org squeeze all

wget -q -O - http://www.dotdeb.org/dotdeb.gpg | sudo apt-key add -

apt-get install redis-server
 
You must log in or register to reply here.
Top
amuck-landowner