• Prepare the chroot directory. It’s recommended to use an extra partition/filesystem for it. I will use /srv/mysql (which is an LVM2 partition with an ext3 filesystem on my system) for the rest of the text.
• Stop MySQL:

  /etc/init.d/mysql stop

• Copy the databases to new location:

  mkdir -p /srv/mysql/var/lib
  cp -a /var/lib/mysql /srv/mysql/var/lib

• Copy this script to /etc/default/mysql-chroot
• Edit /etc/init.d/mysql:
  • Source the mysql-chroot script somewhere at the top:

    …
    test -x /usr/sbin/mysqld || exit 0
    
    . /etc/default/mysql-chroot
    
    SELF=$(cd $(dirname $0); pwd -P)/$(basename $0)
    …

  • Fix the disk space check:

    …
    # check for diskspace shortage
    datadir=`mysqld_get_param datadir`
    if LC_ALL=C BLOCKSIZE= df --portability $CHROOT_DIR$datadir/. | tail -n 1 | awk '{ exit ($4>4096) }'; then
      log_failure_msg "$0: ERROR: The partition with $datadir is too full!"
    …

  • Run setup_chroot right in the start section:

    …
    if mysqld_status check_alive nowarn; then
      echo "...already running."
    else
      setup_chroot
      /usr/bin/mysqld_safe > /dev/null 2>&1 &
    …

  • Somehow /var/run/mysqld/mysqld.pid disappears after each start. We have to create it each time, otherwise the stop command won’t work properly:

    …
    if mysqld_status check_alive warn; then
      log_end_msg 0
      ln -sf $CHROOT_DIR/var/run/mysqld/mysqld.pid \
                     /var/run/mysqld
      # Now start mysqlcheck or whatever the admin wants.
      output=$(/etc/mysql/debian-start)
    …

• In /etc/mysql/debian.cnf, change the two socket lines to:

  socket = /srv/mysql/var/run/mysqld/mysqld.sock

• In /etc/mysql/my.cnf:
  • Change the socket line in the [client] section to:

    socket = /srv/mysql/var/run/mysqld/mysqld.sock

    Don’t change the socket lines in the other sections!

  • Add

    chroot = /srv/mysql

    to the [mysqld] section.

• Prepend /srv/mysql to the log files listed in /etc/logrotate.d/mysql-server
• Make /usr/bin/mysql_upgrade_shell use the chrooted socket. Note: Currently these changes must be made each time mysql gets upgraded because upgrades override this file!

  …
  --password=*) password=`echo "$arg" | sed -e 's/^[^=]*=//'` ;;
  --socket=*) socket=`echo "$arg" | sed -e 's/^[^=]*=//'` ;;
  --ldata=*|--data=*|--datadir=*) DATADIR=`echo "$arg" | sed -e 's/^[^=]*=//'` ;;
  …
  fi
  $bindir/mysql_fix_privilege_tables --silent --user=$user --password=$password --socket=$socket $args
  exit 0
  …
  check_args="--check-upgrade --all-databases --auto-repair --user=$user --password=$password --socket=$socket"
  …
  $bindir/mysql_fix_privilege_tables --silent --user=$user --password=$password --socket=$socket $args
  …

• Start MySQL:

  /etc/init.d/mysql start

• Check /var/log/syslog for errors ;-)

As a result of this move, there are now packages of Sun’s Java for Debian and Ubuntu.
The packaging code is largely based on the code we are using for Blackdown Java for some years. The code is available under the MIT license from the jdk-distros project on java.net. (More info on Tom Marble’s blog.)

I’m glad Sun finally opens Java up a bit after years of restrictive licensing.

After some searching I found out that you apparently need lvm2 2.02.01 or later and devmapper 1.02.02 or later to successfully remove snapshot volumes now. Unfortunately neither of these versions is available for sarge from Debian or backports.org yet, so I had to make my own backports.
As it turned out (see below), it is also necessary to use 2.6.16.12 or to apply the patch from this email to older 2.6.16 versions in order to reliably remove snapshots.

If you are brave enough, you can get the backported packages by adding

deb http://blog.blackdown.de/static/debian/lvm/ sarge main
deb-src http://blog.blackdown.de/static/debian/lvm/ sarge main

to /etc/apt/sources.list.

The repository contains debs for devmapper, dlm, lvm2, and lvm-common. The Release files is signed with my GPG key. If you have a recent apt version, you can authenticate the packages after importing the key with apt-key:

wget http://blog.blackdown.de/static/gpg.asc -O - | \
    sudo apt-key add -

April 15th, 2006: In about 40 backup cycles I’ve seen three lockups with 2.6.16.2 now. Until snapshots get fixed in 2.6.16, I’d recommend to stay with 2.6.15. I’m using 2.6.15.3 again now.

April 24th, 2006: Added note about “dm snapshot: fix kcopyd destructor” patch from Alasdair G Kergon. With this patch snapshots work fine for me again.

May 2nd, 2006: Alasdair G Kergon’s patch has been included in 2.6.16.12.

The situation has not changed much since WordPress 1.5: WordPress 2.0 still does not support HTTPS access to the admin area when the rest of the blog is served via normal HTTP and I still do not like logging in to my server over unencrypted connections, especially not when using public WLANs. Getting around this WordPress limitation requires quite a few steps:

The Goal

All communication involving passwords or authentication cookies should be done over HTTPS connections. wp-login.php and the wp-admin directory should only be accessible over HTTPS.
Normal reading access, as well as comments, tracebacks, and pingbacks still should go over ordinary HTTP.

The Plan

• Add an HTTPS virtual host that forwards requests to the HTTP virtual host
• Modify WordPress to send secure authentication cookies, so cookies never get sent over insecure connections accidentally
• Require a valid certificate on HTTPS clients. That means to log in to WordPress you need both a valid certificate and a valid password. If someone manages to get your password, he still can not login because he does not have a valid certificate.

The Implementation

Note: This documentation assumes a Debian sarge installation with Apache 2. Some things, in particular Apache module related ones, will be different on other systems.
The server used throughout the instructions is example.org/192.0.34.166. The server’s DocumentRoot is /blog and WordPress resides in /blog/wp. The value of WordPress’ home option is ‘http://example.org&/#8217; and the value of its site_url option is ‘http://example.org/wp’.

• Prepare the SSL certificates:
  • Generate your own certificate authority (CA) if you don’t have one already (I’m using the makefile from OpenSSL Certificate Authority Setup for managing mine) and import it into your browser.
  • Generate a certificate for the SSL server and certify it with your private CA.
  • Generate a certificate for your browser and certify it with your private CA. Most browsers expect a PKCS#12 file, so generate one with

    $ openssl pkcs12 -export -clcerts \
        -in blogclient.cert \
        -inkey blogclient.key \
        -out blogclient.p12

    Then import blogclient.p12 into your browser.

• Make WordPress SSL-ready:
  Apply this patch to the WordPress code. It makes the following changes:
  • Use secure authentication cookies in wp_setcookie()
  • Make check_admin_referer() work with HTTPS URLs
  • Use HTTPS URLs for notification mails
  • Use HTTPS URLS for redirects to wp-login.php
  • Only allow XML-RPC logins from the local host (ie. the HTTPS proxy)
  • Add the Mark-as-Spam feature from trunk

  The patch is against svn version 3825 of WordPress (ie. WordPress 2.0.3), when you apply it to a newer version, you will likely get some harmless ‘Hunk succeeded’ message. If you are getting ‘Hunk FAILED’ message, just send me note and I’ll update the patch.

• Enable the necessary Apache modules:
  • Install mod_proxy_html. It will be used to replace absolute ‘http://example.org&/#8217; HTTP URLs in the WordPress output with ‘https://example.org&/#8217; HTTPS URLs:

    $ aptitude install libapache2-mod-proxy-html

    The module gets enabled automatically after installation.

  • Enable mod_proxy and mod_ssl

    $ a2enmod proxy
    $ a2enmod ssl

    Debian provides sane default configurations for both modules. You might want to take a look at the configuration files (ssl.conf and proxy.conf) nevertheless.
    I have changed SSLCipherSuite to

    TLSv1:SSLv3:!SSLv2:!aNULL:!eNULL:!NULL:!EXP:!DES:!MEDIUM:!LOW:@STRENGTH

    in ssl.conf in order to just allow TLS v1 and SSL v3 ciphers which provide strong encryption and authentication (see ciphers(1)).

  • If you are compressing WordPress output (that is if you enabled the ‘WordPress should compress articles (gzip) if browsers ask for them’ option) then also enable mod_headers:

    $ a2enmod headers

• Configure Apache to listen on the HTTPS port

  $ cat > /etc/apache2/conf.d/ssl.conf << EOF
  <IfModule mod_ssl.c>
  	Listen 443
  </IfModule>
  EOF

• Modify the blog virtual host to limit access to wp-login.php and wp-admin to the local host. Also completely deny access to files which should never be accessed directly. Here is an example: 10-wp2-example.org
• Now setup the HTTPS virtual server: 20-wp2-example.org-ssl
  If you are compressing WordPress output you have to enable the RequestHeader line.
• Enable the site and restart Apache

  $ a2ensite 20-blog-ssl
  $ /etc/init.d/apache2 restart

• Remove the old WP cookies from your browser
• Test the new setup!

February 1st, 2006: wp2-ssl.patch updated for WordPress 2.0.1

March 11st, 2006: WordPress 2.0.2 has been released, fixing some security issues. The HTTPS patch still applies fine to that version.

March 19th, 2006: Updated wp2-ssl.patch. Changes: Fix bug in list-manipulation.php, use HTTPS for ‘Login’ and ‘Register’ links, backport ‘Mark-as-Spam’ feature from trunk

May 1st, 2006: WordPress 2.0.3 has been released. Here is the updated wp2-ssl.patch.

July 29th, 2006: WordPress 2.0.4 has been released, fixing some security issues. Here is an updated version of the wp2-ssl.patch.

January 12st, 2007: wp2-ssl.patch updated for 2.0.6 and 2.0.7-RC1

January 15st, 2007: WordPress 2.0.7 has been released. The patch still applies fine to that version.

The lack of security support was one of the main problems with “testing”. You had to pull security fixes from “unstable” or even build your own packages to keep it secure.

I hope they have the manpower to keep up with security issues. Debian’s main security team, which only provides updates for the “stable” distribution, had some problems over the last months.

The official sarge installer still uses a 2.4 kernel by default but includes a 2.6 kernel that can be used by booting with "install26" or "expert26". But even that kernel, 2.6.8, is too old for the Inspiron 9300. It still doesn’t recognize the hard disk.

Ubuntu’s installer, which uses a 2.6.11 kernel, works fine on the machine. Although Ubuntu is a nice distribution, I like pure Debian better. Unfortunately I wasn’t able to find any 2.6.11 based Debian installer on the net, even a question on debian-boot yielded nothing.

Anyhow, I finally had the time to build one myself:
debian-2.6.11-i386-businesscard.iso (GPG signature)

The image is basically a sarge businesscard ISO with a 2.6.11 kernel from Debian testing instead of the original 2.6.8 kernel.

Unlike with Ubuntu, installation on the Inspiron 9300 still doesn’t work out of the box but with a few tricks I was able to install Debian sarge:

• Boot with expert26
• When the installer starts up, switch to the second console (Alt-F2) and enter these commands:

  ~ # modprobe ide_generic
  ~ # modprobe ata_piix

  Without this the installer won’t find the CD-ROM.

• If network configuration via DHCP fails, just retry — worked for me
• When asked what version of Debian you would like to install, choose stable. Installing testing or unstable directly doesn’t work.
• It doesn’t matter which kernel you choose to install, we have to replace it with a 2.6.11 kernel later anyway
• Just before the first reboot, that means right after the installer ejects the CD-ROM, switch back to console two. Now download and install the latest available Debian kernel. I’ve used 2.6.11-1-686:

  ~ # mount -t proc proc /target/proc
  ~ # chroot /target
  sh-2.05b# cd /root
  sh-2.05b# wget http://blog.blackdown.de/static/debian/kernel-image-2.6.11-1-686_2.6.11-7_i386.deb
  sh-2.05b# dpkg -i kernel-image-2.6.11-1-686_2.6.11-7_i386.deb
  …
  sh-2.05b# exit
  ~ # umount /target/proc

• Reboot (using the kernel just installed) and complete the installation
• Upgrade to testing or unstable
• Build a custom kernel (2.6.12 or newer). It’s probably a good idea to include some additional libata patches. To get the DVD drive working you have to apply this patch.

deb ftp://ftp.tux.org/java/debian/ sarge non-free

to your /etc/apt/sources.list.

Upgrading is recommended as 1.4.2-02 contains an important security fix.

As one can guess from the look of this site, I’m using WordPress as my blog engine. At this time WordPress does not support HTTPS access to the admin area when the rest of the blog is served via normal HTTP. This is a bit unfortunate. I do not like logging in to my server over unencrypted connections, especially not when using public WLANs. Getting around this WordPress limitation requires quite a few steps:

The Goal

All communication involving passwords or authentication cookies should be done over HTTPS connections. wp-login.php and the wp-admin directory should only be accessible over HTTPS.
Normal reading access, as well as comments, tracebacks, and pingbacks still should go over ordinary HTTP.

The Plan

• Add an HTTPS virtual host that forwards requests to the HTTP virtual host
• Modify WordPress to send secure authentication cookies, so cookies never get sent over insecure connections accidentally
• Require a valid certificate on HTTPS clients. That means to log in to WordPress you need both a valid certificate and a valid password. If someone manages to get your password, he still can not login because he does not have a valid certificate.

The Implementation

Note: This documentation assumes a Debian sarge installation with Apache 2. Some things, in particular Apache module related ones, will be different on other systems.
The server used throughout the instructions is example.org/192.0.34.166. The server’s DocumentRoot is /blog and WordPress resides in /blog/wp. The value of WordPress’ home option is ‘http://example.org&/#8217; and the value of its site_url option is ‘http://example.org/wp’.

• Prepare the SSL certificates:
  • Generate your own certificate authority (CA) if you don’t have one already (I’m using the makefile from OpenSSL Certificate Authority Setup for managing mine) and import it into your browser.
  • Generate a certificate for the SSL server and certify it with your private CA.
  • Generate a certificate for your browser and certify it with your private CA. Most browsers expect a PKCS#12 file, so generate one with

    $ openssl pkcs12 -export -clcerts \
        -in blogclient.cert \
        -inkey blogclient.key \
        -out blogclient.p12

    Then import blogclient.p12 into your browser.

• Make WordPress SSL-ready:
  Apply this patch to the WordPress code. It makes the following changes:
  • Use secure authentication cookies in wp_setcookie()
  • Make check_admin_referer() working with HTTPS URLs
  • Disable login over XML-RPC
• Enable the necessary Apache modules:
  • Install mod_proxy_html. It will be used to replace absolute ‘http://example.org&/#8217; HTTP URLs in the WordPress output with ‘https://example.org&/#8217; HTTPS URLs:

    $ aptitude install libapache2-mod-proxy-html

    The module gets enabled automatically after installation.

  • Enable mod_proxy and mod_ssl

    $ a2enmod proxy
    $ a2enmod ssl

    Debian provides sane default configurations for both modules. You might want to take a look at the configuration files (ssl.conf and proxy.conf) nevertheless.

  • If you are compressing WordPress output (that is if you enabled the ‘WordPress should compress articles (gzip) if browsers ask for them’ option) then also enable mod_headers:

    $ a2enmod headers

• Configure Apache to listen on the HTTPS port

  $ cat > /etc/apache2/conf.d/ssl.conf << EOF
  <IfModule mod_ssl.c>
  	Listen 443
  </IfModule>
  EOF

• Modify the blog virtual host to limit access to wp-login.php and wp-admin to the local host. Also completely deny access to files which should never be accessed directly. Here is an example: 10-example.org
• Now setup the HTTPS virtual server: 20-example.org-ssl
  If you are compressing WordPress output you have to enable the RequestHeader line.
• Enable the site and restart Apache

  $ a2ensite 20-blog-ssl
  $ /etc/init.d/apache2 restart

• Remove the old WP cookies from your browser
• Test the new setup!

My exim configuration uses three different cyrus_sasl authenticators and each exim invocation resulted in three of these OTP warnings because exim calls sasl_listmech() for each configured authenticator. It doesn’t specify a limiting mech_list, that means SASL will test which of all installed mechs actually can be used for authentication. Debian’s SASL package includes libotp.so, so it also tries to use OTP which is not configured on my system.

There are two ways to get rid off the warnings:

• Remove /usr/lib/sasl2/libotp.*. You’ll have to do this after each upgrade of the libsasl2-modules package.
• Rebuild exim with this patch. The patch specifies a limiting mech_list option for SASL. This limits sasl_listmech() to the mechs used in the exim configuration. Other mechs won’t be tried anymore.

May 3rd, 2005: A slightly modified version of the patch has been integrated into Exim CVS and will be included in the next Debian release of exim4 (see Debian bug #299743)

That means libc.so.6 is linked statically now. But glibc’s getpwnam and getpwuid implementations still need the shared libraries. The needed libraries must be copied into the chroot because mysqld calls those functions after calling chroot. I’ve updated the mysql-chroot script accordingly.
(The rest of the chroot setup procedure still works as described in Chrooting MySQL on Debian.)

By the way, I’ve filed a wishlist bug at Debian’s BTS (#299265). mysqld should do all /etc/passwd lookups before calling chroot. That way chrooting would work without $CHROOT/etc/passwd and with copying any libraries into the chroot. That’s how Apache and Bind 9 do it.

March 17th, 2005: Debian has removed the -all-static flag again. I’m leaving the additional bits in the chroot script however, just in case the maintainers decide to add the flag again.

<VirtualHost 127.0.0.2:80>
        ServerName JustForOpeningSyslog
        Redirect permanent / http://127.0.0.1/
        ErrorLog syslog
</VirtualHost>

Now apache calls openlog(3) with LOG_NDELAY before being chrooted by libapache2-mod-chroot, and libapache2-mod-php4’s syslog(3) calls work just fine.
(Idea stolen from syslog(3) and chroot(2).)

But I’ve found no guide for chrooting MySQL, so here’s my short recipe:

• Prepare the chroot directory. It’s recommended to use an extra partition/filesystem for it. I will use /srv/mysql (which is an LVM2 partition with an ext3 filesystem on my system) for the rest of the text.
• Stop MySQL:

  /etc/init.d/mysql stop

• Copy the databases to new location:

  mkdir -p /srv/mysql/var/lib
  cp -a /var/lib/mysql /srv/mysql/var/lib

• Copy this script to /etc/default/mysql-chroot
• Edit /etc/init.d/mysql:
  • Source the mysql-chroot script somewhere at the top:

    …
    test -x /usr/sbin/mysqld || exit 0
    
    . /etc/default/mysql-chroot
    
    SELF=$(cd $(dirname $0); pwd -P)/$(basename $0)
    …

  • Run setup_chroot right in the start section:

    …
    if mysqld_status check_alive nowarn; then
      echo "...already running."
    else
      setup_chroot
      /usr/bin/mysqld_safe > /dev/null 2>&1 &
    …

  • Somehow /var/run/mysqld/mysqld.pid disappears after each start. We have to create it each time, otherwise the stop command won’t work properly:

    …
    if mysqld_status check_alive warn; then
      echo "."
      ln -sf $CHROOT_DIR/var/run/mysqld/mysqld.pid \
                     /var/run/mysqld
      # Now start mysqlcheck or whatever the admin wants.
      /etc/mysql/debian-start
    …

• In /etc/mysql/debian.cnf, change the socket line to:

  socket = /srv/mysql/var/run/mysqld/mysqld.sock

• In /etc/mysql/my.cnf:
  • Change the socket line in the [client] section to:

    socket = /srv/mysql/var/run/mysqld/mysqld.sock

    Don’t change the socket lines in the other sections!

  • Add

    chroot = /srv/mysql

    to the [mysqld] section.

• Prepend /srv/mysql to the log files listed in /etc/logrotate.d/mysql-server
• Start MySQL:

  /etc/init.d/mysql start

• Check /var/log/syslog for errors ;-)

March 13th, 2005: I’ve updated the script for newer Debian packages, see Updated MySQL Chroot Script for more information.

July 30th, 2006: These modifications still work fine on the current stable Debian release (3.1, “sarge”). The mysql packages in the testing (“etch”) and unstable (“sid”) distributions of Debian need a few additional changes, I’ll post an updated guide in a few days.

December 30th, 2006: I’ve made an updated guide on how to chroot more recent MySQL packages on Debian and Ubuntu

• /etc/exim4/exim.pl:
  Don’t forget to change ppp0 to the interface you want to handle!

  #! /usr/bin/perl
  
  # Requires libio-interface-perl
  
  use strict;
  use IO::Socket;
  use IO::Interface;
  
  sub get_remote_helo_data()
  {
      my $s = IO::Socket::INET->new(Proto => 'udp');
      my $addr = inet_aton($s->if_addr('ppp0'));
      my $hostname = gethostbyaddr($addr, AF_INET);
  
      $hostname = '' if (!$hostname);
  
      return $hostname;
  }
  

• /etc/exim4/conf.d/main/50_exim4-localconfig_perl:

  #main/50_exim4-localconfig_perl
  perl_at_start = true
  perl_startup = do '/etc/exim4/exim.pl'
  

• Add the following code to the appropriate transport, e.g. remote_smtp_smarthost:

  helo_data = \
    ${if >{${strlen:${perl{get_remote_helo_data}}}}{0} \
                   {${perl{get_remote_helo_data}}} \
                   {$primary_hostname}}