amuck-landowner

Using varnish as a http cache

vRozenSch00n

vRozenSch00n

Active Member
@wlanboy & everybody in this thread, I need your suggestion. This is my /etc/sysconfig/varnish setting:

VARNISH_RUN_USER=varnish
VARNISH_RUN_GROUP=varnish

# Maximum number of open files (for ulimit -n)
NFILES=131072
# Locked shared memory (for ulimit -l)
MEMLOCK=82000
# Maximum number of threads (for ulimit -u)
NPROCS="unlimited"
# Maximum size of corefile (for ulimit -c). Default in Fedora is 0
# DAEMON_COREFILE_LIMIT="unlimited"

RELOAD_VCL=1

# # Should probably change this
VARNISH_VCL_CONF=/etc/varnish/default.vcl

# # Not setting VARNISH_LISTEN_ADDRESS makes Varnish listen on all IPs on this box (Both IPv4 and IPv6 if available). Set manually to override this.
# VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=80

# # Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=localhost
VARNISH_ADMIN_LISTEN_PORT=6082

# # Shared secret file for admin interface
VARNISH_SECRET_FILE=/etc/varnish/secret

# # The minimum number of worker threads to start
VARNISH_MIN_THREADS=1

# # The Maximum number of worker threads to start
VARNISH_MAX_THREADS=1000

# # Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120

# Best option is malloc if you can. malloc will make use of swap space smartly if you have it and need it.
VARNISH_STORAGE_TYPE=malloc
#VARNISH_STORAGE_TYPE=file

VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin

# # Cache file size: in bytes, optionally using k / M / G / T suffix, in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=384M

VARNISH_STORAGE="${VARNISH_STORAGE_TYPE},${VARNISH_STORAGE_SIZE}"
#VARNISH_STORAGE="${VARNISH_STORAGE_TYPE},${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"

# # Default TTL used when the backend does not specify one
VARNISH_TTL=120

VARNISH_POOL_DELAY=2
VARNISH_POOL_CPU=2
#<800 / Number of CPU cores>
VARNISH_POOL_MIN=400
VARNISH_POOL_MAX=4000
VARNISH_SESSION_LINGER=50
VARNISH_SESS_WORKSPACE=262144

# # DAEMON_OPTS is used by the init script. If you add or remove options, make
# # sure you update this section, too.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
-f ${VARNISH_VCL_CONF} \
-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
-t ${VARNISH_TTL} \
-w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
-u ${VARNISH_RUN_USER} -g ${VARNISH_RUN_GROUP} \
-S ${VARNISH_SECRET_FILE} \
-p thread_pool_add_delay=${VARNISH_POOL_DELAY} \
-p thread_pools=${VARNISH_POOL_CPU} \
-p thread_pool_min=${VARNISH_POOL_MIN} \
-p thread_pool_max=${VARNISH_POOL_MAX} \
-p session_linger=${VARNISH_SESSION_LINGER} \
-p sess_workspace=${VARNISH_SESS_WORKSPACE} \
-s ${VARNISH_STORAGE}"
Click to expand...
Is it OK?

The backend uses Apache + xcache (Kloxo)

When I check with for example: curl -I http://mywebsite.com

The first attempt always shows: X-Cache: MISS

Is that normal?
 
wlanboy

wlanboy

Content Contributer
Depends on the app behind.


Things like Cache-Control: no-cache, must-revalidate


and http authentification can bypass the cache.
 
vRozenSch00n

vRozenSch00n

Active Member
Here you go :

http://www.fashionway.be/

backend default {
.host = "199.195.xxx.xxx";
.port = "8080";
.max_connections = 50;
.connect_timeout = 4.0s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
}

acl purge {
"199.195.xxx.xxx";
}

### Called when a client request is received
sub vcl_recv {

# In the event of a backend overload (HA!), serve stale objects for up to two minutes
set req.grace = 2m;

# First call our identify_device subroutine to detect the device
# call identify_device;

# Add a unique header containing the client address
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;
# set req.http.X-Forwarded-For = req.http.rlnclientipaddr;

### restart logic, this will redefine the backends if vcl_restart has been triggered
if (req.restarts == 0) {
set req.backend = default;
} else if (req.restarts == 1) {
set req.backend = default;
} else if (req.restarts == 2) {
set req.backend = default;
} else {
set req.backend = default;
}

### do not cache these files:

if (req.request == "GET" && req.url ~ "\.(cfm)") {
return(pass);
}

### always cache these items:

if (req.request == "GET" && req.url ~ "\.(js)") {
return(lookup);
}

## images
if (req.request == "GET" && req.url ~ "\.(gif|jpg|jpeg|bmp|png|tiff|tif|ico|img|tga|wmf)$") {
return(lookup);
}

## various other content pages
if (req.request == "GET" && req.url ~ "\.(css|htm|html)$") {
return(lookup);
}

## multimedia
if (req.request == "GET" && req.url ~ "\.(svg|swf|ico|mp3|mp4|m4a|ogg|mov|avi|wmv|flv)$") {
return(lookup);
}

## xml
if (req.request == "GET" && req.url ~ "\.(xml)$") {
return(lookup);
}

### do not cache these rules:
if (req.request != "GET" && req.request != "HEAD") {
return(pipe);
}
if (req.http.Authenticate || req.http.Authorization) {
return(pass);
}

### don't cache authenticated sessions
if (req.http.Cookie && req.http.Cookie ~ "(GuppYUser|GuppyAdmin)") {
return(pipe);
}

if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico|png)(\?.*|)$") {
unset req.http.cookie;
set req.url = regsub(req.url, "\?.*$", "");
}


### if there is a purge make sure its coming from $localhost

if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
return(lookup);
}

### parse accept encoding rulesets to make it look nice
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(gif|jpg|jpeg|swf|flv|mp3|mp4|pdf|ico|png|gz|tgz|bz2)(\?.*|)$") {
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unkown algorithm
remove req.http.Accept-Encoding;
}
}

### if it passes all these tests, do a lookup anyway;
return(lookup);

}

### Called when the requested object has been retrieved from the backend, or the request to the backend has failed
sub vcl_fetch {
# In the event of a backend overload (HA!), serve stale objects for up to two minutes
set req.grace = 2m;

# if (req.url ~ "admin" || req.url ~ "xmlrpc.php" || req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js)(\?[a-z0-9]+)?$") {
# return (hit_for_pass);
# }
if ( (!(req.url ~ "admin")) || (req.request == "GET") ) {
# beresp == Back-end response from the web server.
unset beresp.http.set-cookie;
set beresp.ttl = 1h;
}
if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico|png)(\?.*|)$") {
set beresp.ttl = 30d;
}
if (beresp.status >= 400) {
set beresp.ttl = 5s;
set beresp.grace =0s;
}
}


### Called when an object is in the cache, its a hit.
sub vcl_hit {
if (req.request == "PURGE") {
set obj.ttl = 0s;
error 200 "Purged.";
}
return (deliver);
}

### Called when the requested object was not found in the cache
sub vcl_miss {
if (req.request == "PURGE") {
error 404 "Not in cache.";
}
}

## Called before a cached object is delivered to the client
sub vcl_deliver {
set resp.http.X-Served-By = server.hostname;
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
set resp.http.X-Cache-Hits = obj.hits;
} else {
set resp.http.X-Cache = "MISS";
}

return (deliver);
}

sub vcl_pipe {
set bereq.http.connection = "close";
return (pipe);
}
Click to expand...
 
Last edited by a moderator:
wlanboy

wlanboy

Content Contributer
vRozenSch00n said:
Here you go :
Click to expand...
So you are handling wordpress too.

Good to see someone doing this because I need an second opinion about "logged in users".

Code: 
if( req.url ~ "^/wp-(login|admin)" || req.http.Cookie ~"(wp-postpass|wordpress_logged_in|comment_author_)" ){
    return (pass);
}
 
E

ekobayu

New Member
hello everyone,

I have some question, sorry if my question is out of topic.

I'm want to create a GUI panel configuration for varnish.

Any advice for me ? is better using php or jquery ?

sorry for my bad english,
 
M

marlencrabapple

New Member
ekobayu said:
hello everyone,

I have some question, sorry if my question is out of topic.

I'm want to create a GUI panel configuration for varnish.

Any advice for me ? is better using php or jquery ?

sorry for my bad english,
Click to expand...
Its not a matter of or. You'll be forced to use PHP, Python, Perl, Ruby, etc., for the backend.
 
You must log in or register to reply here.
Top
amuck-landowner