Wordpress's WP-Super-Cache's Super Cache with nginx
(Apologies for the triple-layer title, but it's a specific subject involving a badly named plugin.)
This has been explained before (the progenitor for most other examples on the net seems to be this forum post), but the solution was ugly and slightly incomplete. nginx's lack of a one-line RewriteCond equivalent means there will never be an elegant solution, but I think I've come up with something clearer.
First, background. WP Super Cache has two levels of caching:
- "WP Cache". Whenever Wordpress's index.php renders a page, a copy of the page output is stored in
/blog/wp-content/cache(and themetasubfolder). For future requests for the same page, this cached copy is served byindex.php. The good: subsequent requests don't hit the database or re-run your badly coded widgets for every visitor. The bad: PHP still runs for every request. - "Super Cache". As well as a copy of page output being stored as per above, a copy is also stored in
/blog/wp-content/supercache, in a structure that mirrors your blog's URL hierarchy. With clever use of rewrite rules at your webserver layer, you can entirely skip loading PHP & Wordpress for any request that a cached file has been created for.
The WP Cache layer always works. The rest of this post is about making use of the Super Cached files with your shiny nginx server. For reference, the Apache rules are here. This nginx code follows the same order and structure, but has some differences. Read:
location /blog {
gzip_static on;
Aside: gzip_static requires an nginx configured with --with-http_gzip_static_module. If your build isn't, and you don't want to compile your own, just remove this directive. Instead of serving pre-compressed Super Cache files to clients that support compression, nginx will compress them on the fly (like normal).
set $supercache "";
if ($request_method = GET) {
set $supercache "${supercache}G";
}
if ($args = "") {
set $supercache "${supercache}A";
}
if ($http_cookie !~ (comment_author_|wordpress_logged_in|wp-postpass_)) {
set $supercache "${supercache}C";
}
if (-f $document_root/blog/wp-content/cache/supercache/$http_host$request_uri/index.html) {
set $supercache "${supercache}F";
}
# If we met all the conditions, serve the supercached file
if ($supercache = GACF) {
rewrite ^ /blog/wp-content/cache/supercache/$http_host$request_uri/index.html break;
}
# Otherwise pass to wordpress as normal
if (!-e $request_filename) {
rewrite ^ /blog/index.php last;
}
}
# The cache files should not be directly accessible to clients
location /blog/wp-content/cache { internal; }
# Configure the PHP backend as per normal
location ~ (\.php$) {
include fastcgi_params;
if (-e $request_filename) {
fastcgi_pass unix:/tmp/nginx-php-fastcgi.sock;
}
}
Done! If you have problems, three pointers:
- WP Super Cache has a very big settings page. You can set them as you like mostly, but make sure you set this and this (if you're using gzip_static).
- Check the bottom of the source of your pages to see if a page was server from the cache, and if so, whether it was served from the Super Cache.
- If you need to troubleshoot, make liberal use of the logging facility that WP Super Cache implements.
Cross-compiling x264 for win32 on Ubuntu Linux
The total lack of documentation on compiling x264 (and dependencies) for win32 on a linux32 system is henceforth rectified. This guide assumes you are using Ubuntu 9.10 and the packaged version of mingw32. Newer versions of the below packages might require additional/less wrangling.
Required packages in the base system:
sudo apt-get install pkg-config yasm subversion cvs git-core mingw32
Create the basic tree for installing win32-compatible dependancies to:
mkdir -p ~/win32-x264/{src,lib,include,share,bin}
Place this helper script at ~/win32-x264/mingw and chmod +x it:
#!/bin/sh export CC=i586-mingw32msvc-gcc export CXX=i586-mingw32msvc-g++ export CPP=i586-mingw32msvc-cpp export AR=i586-mingw32msvc-ar export RANLIB=i586-mingw32msvc-ranlib export ADD2LINE=i586-mingw32msvc-addr2line export AS=i586-mingw32msvc-as export LD=i586-mingw32msvc-ld export NM=i586-mingw32msvc-nm export STRIP=i586-mingw32msvc-strip export PATH="/usr/i586-mingw32msvc/bin:$PATH" export PKG_CONFIG_PATH="$HOME/win32-x264/lib/pkgconfig/" exec "$@"
Now to install pthread & zlib:
cd ~/win32-x264/src wget -qO - ftp://sourceware.org/pub/pthreads-win32/pthreads-w32-2-8-0-release.tar.gz | tar xzvf - cd pthreads-w32-2-8-0-release make GC-static CROSS=i586-mingw32msvc- cp libpthreadGC2.a ../../lib cp *.h ../../include
cd ~/win32-x264/src wget -qO - http://zlib.net/zlib-1.2.4.tar.gz | tar xzvf - cd zlib-1.2.4 ../../mingw ./configure # Remove references to "-lc" from the Makefile (tells GCC to link output with libc, which is implied anyway, and explicit declaration causes a script error) sed -i"" -e 's/-lc//' Makefile make DESTDIR=../.. make install prefix=
Installing FFmpeg:
cd ~/win32-x264/src svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg cd ffmpeg # Delete references to -Wmissing-prototypes, a GCC warning that fails when cross-compiling sed -i"" -e '/missing-prototypes/d' configure ./configure \ --target-os=mingw32 --cross-prefix=i586-mingw32msvc- --arch=x86 --prefix=../.. \ --enable-memalign-hack --enable-gpl --enable-avisynth --enable-postproc --enable-runtime-cpudetect \ --disable-encoders --disable-muxers --disable-network --disable-devices make make install
Installing FFmpegsource:
cd ~/win32-x264/src svn checkout http://ffmpegsource.googlecode.com/svn/trunk/ ffms cd ffms ../../mingw ./configure --host=mingw32 --with-zlib=../.. --prefix=$HOME/win32-x264 ../../mingw make make install
Installing GPAC:
Special thanks to the GPAC dev who kindly assisted me in beating the terrible configure/Makefile scripts into shape.
cd $HOME/win32-x264/src # Create a CVS auth file on your machine cvs -d:pserver:anonymous@gpac.cvs.sourceforge.net:/cvsroot/gpac login cvs -z3 -d:pserver:anonymous@gpac.cvs.sourceforge.net:/cvsroot/gpac co -P gpac cd gpac chmod +rwx configure src/Makefile # Hardcode cross-prefix sed -i'' -e 's/cross_prefix=""/cross_prefix="i586-mingw32msvc-"/' configure ../../mingw ./configure --static --use-js=no --use-ft=no --use-jpeg=no --use-png=no --use-faad=no --use-mad=no --use-xvid=no --use-ffmpeg=no --use-ogg=no --use-vorbis=no --use-theora=no --use-openjpeg=no --disable-ssl --disable-opengl --disable-wx --disable-oss-audio --disable-x11-shm --disable-x11-xv --disable-fragments--use-a52=no --disable-xmlrpc --disable-dvb --disable-alsa --static-mp4box --extra-cflags="-I$HOME/win32-x264/include -I/usr/i586-mingw32msvc/include" --extra-ldflags="-L$HOME/win32-x264/lib -L/usr/i586-mingw32msvc/lib" # Fix pthread lib name sed -i"" -e 's/pthread/pthreadGC2/' config.mak # Add extra libs that are required but not included sed -i"" -e 's/-lpthreadGC2/-lpthreadGC2 -lwinmm -lwsock32 -lopengl32 -lglu32/' config.mak make # Make will fail a few commands after building libgpac_static.a (i586-mingw32msvc-ar cr ../bin/gcc/libgpac_static.a ...). That's fine, we just need libgpac_static.a cp bin/gcc/libgpac_static.a ../../lib/ cp -r include/gpac ../../include/
Building x264:
cd ~/win32-x264/src git clone git://git.videolan.org/x264.git cd x264 ./configure --cross-prefix=i586-mingw32msvc- --host=i586-pc-mingw32 --extra-cflags="-I$HOME/win32-x264/include" --extra-ldflags="-L$HOME/win32-x264/lib" make
Leave to cool for 15 minutes. Serves four.
Changelog:
- 20100518: Updated ffmpeg configure args. ffms build needs mingw wrapper. Add cvs to required packages.
Tarsnap backups on Windows and Linux
For the past month or so I've been running Tarsnap to backup my home desktop and remote servers. Tarsnap is an online (payware) backup system that's written by Colin Percival, and is probably the most technically impressive backup system I've seen. As well as that, it's damn cheap.
What's so great about Tarsnap? Here's three things:
- Snapshot backups. Every backup you create with tarsnap is a standalone 'snapshot' of your data, totally independent of all other snapshots.
- Backup deduplication. So your backup includes a 300mb log file that gets a few megs of data written to it every day. Tarsnap recognises this, and for each new snapshot only the changed data has to be uploaded and stored.
- Crazy Ass Security. While mild-mannered Colin Percival works on Tarsnap during the day, by night he's the FreeBSD Security Officer. He's been there since 2005, so must be doing something right. Plus, check out this page. How can something with that many mentions of AES, SHA and RSA be insecure?! :-)
This post isn't a HOWTO on setting up Tarsnap, there's a comprehensive tutorial on the subject already. This post is just to document how I use Tarsnap, on both Linux and Windows.
Linux
My Linux servers perform a simple daily backup of everything in certain folders. Technically, I could simply backup "/" and exclude the directories I don't want, but smaller archives are faster to restore from, and there's no monetary penalty for having more snapshots. This is my script:
#!/bin/bash for dir in $(cat /root/tarsnap-dirs) ; do nice tarsnap -c -f $(hostname -s)-$(date -u +%Y%m%d-%H%M%S)-$(echo $dir | tr -d '/') --one-file-system -C / $dir done # Delete backups more than n days old # n=10 # tarsnap --list-archives | sort | cut -d- -f1-2 | uniq | tail -n +$n > /tmp/temp.$$ # tarsnap --list-archives | fgrep -f /tmp/temp.$$ | while read archive ; do # echo Deleting $archive # tarsnap -d -f $archive # done # rm /tmp/temp.$$ tarsnap --print-stats
(This script runs from crontab, so the output gets mailed to me daily. The email is sent to the address specified as the MAILTO variable in /etc/crontab.)
My .tarsnaprc looks like this:
keyfile /root/tarsnap-key-abraxo.key cachedir /root/tarsnap-cache/ exclude /root/tarsnap-cache/ humanize-numbers
Each day, a backup of each folder listed in the file 'tarsnap-dirs' is created, with names like: 'bluebottle-20091209-200001-homeaj'. There's commented out support for deleting old archives too, but my monthly costs are so low I keep everything.
Windows
My Windows setup is basically identical, but since there's no native Windows (or msys) support for Tarsnap, you have to make do with Cygwin. Explaining how to install Cygwin is far beyond the scope of this document, but it's pretty simple. Apart from the standard Tarsnap dependancies, you will probably also want to install 'ssmtp', which will let you email Tarsnap's output to yourself like Unix cron does.
(You can generate ssmtp's config file by running 'ssmtp-config'.)
Again, my script:
#!/bin/bash log=/tmp/tarsnap.log.$$ cat <$log From: Alex Jurkiewicz Subject: tarsnap run $(hostname)-$(date +%Y%m%d) To: Alex Jurkiewicz EOF tarsnap -c -f $(hostname)-$(date +%Y%m%d-%H%M%S)-homeaj -C /home/ aj >>$log 2>&1 tarsnap -c -f $(hostname)-$(date +%Y%m%d-%H%M%S)-CUsersAlex -C /cygdrive/c/Users/ Alex >>$log 2>&1 tarsnap --print-stats >>$log 2>&1 cat $log | /usr/sbin/ssmtp.exe alex@bluebottle.net.au rm $log
Because I want to exclude a lot of directories on Windows, I put these in my ~/.tarsnaprc file:
exclude Desktop/ exclude AppData/Local/Temp/
And so on. So there you have it. Simple, painless backups with my favourite new toy, Tarsnap.
Comment (0)Ubuntu 9.10 nvidia-settings - xorg.conf save fail
Ubuntu 9.10 creates a (relatively) minimal xorg.conf file compared to what older Ubuntus have. This causes the nvidia-settings app to be unable to parse the file (and therefore save a new version), specifically because of this missing line from the "Screen" section:
Device "Default Device"
If you add this line to your default screen configuration in /etc/X11/xorg.conf, nvidia-settings should be happy again.
Gripe: didn't anyone pick this up during testing?
Comment (1)