Picture of Jürgen Kreileder

Archive for the ‘Linux Kernel’ Category

LVM Snapshots With Debian Sarge and Linux 2.6.16

I have upgraded this server to kernel 2.6.16.2. The next backup cycle resulted in a minor disaster: The backup process deadlocked at removing the first LVM2 snapshot and the snapshot source volumes were blocking write accesses. A cleanup shutdown was impossible and I had to hard-reset the machine.

After some searching I found out that you apparently need lvm2 2.02.01 or later and devmapper 1.02.02 or later to successfully remove snapshot volumes now. Unfortunately neither of these versions is available for sarge from Debian or backports.org yet, so I had to make my own backports.
As it turned out (see below), it is also necessary to use 2.6.16.12 or to apply the patch from this email to older 2.6.16 versions in order to reliably remove snapshots.

If you are brave enough, you can get the backported packages by adding

deb http://blog.blackdown.de/static/debian/lvm/ sarge main
deb-src http://blog.blackdown.de/static/debian/lvm/ sarge main

to /etc/apt/sources.list.

The repository contains debs for devmapper, dlm, lvm2, and lvm-common. The Release files is signed with my GPG key. If you have a recent apt version, you can authenticate the packages after importing the key with apt-key:

wget http://blog.blackdown.de/static/gpg.asc -O - | \
    sudo apt-key add -

April 15th, 2006: In about 40 backup cycles I’ve seen three lockups with 2.6.16.2 now. Until snapshots get fixed in 2.6.16, I’d recommend to stay with 2.6.15. I’m using 2.6.15.3 again now.

April 24th, 2006: Added note about “dm snapshot: fix kcopyd destructor” patch from Alasdair G Kergon. With this patch snapshots work fine for me again.

May 2nd, 2006: Alasdair G Kergon’s patch has been included in 2.6.16.12.

Debian Installer With Kernel 2.6.11

As mentioned recently, Debian Sarge’s installer doesn’t work on my Dell Inspiron 9300. I like Debian but I think it’s a shame that the sarge installer was already outdated on the day of its release.

The official sarge installer still uses a 2.4 kernel by default but includes a 2.6 kernel that can be used by booting with "install26" or "expert26". But even that kernel, 2.6.8, is too old for the Inspiron 9300. It still doesn’t recognize the hard disk.

Ubuntu’s installer, which uses a 2.6.11 kernel, works fine on the machine. Although Ubuntu is a nice distribution, I like pure Debian better. Unfortunately I wasn’t able to find any 2.6.11 based Debian installer on the net, even a question on debian-boot yielded nothing.

Anyhow, I finally had the time to build one myself:
debian-2.6.11-i386-businesscard.iso (GPG signature)

The image is basically a sarge businesscard ISO with a 2.6.11 kernel from Debian testing instead of the original 2.6.8 kernel.

Unlike with Ubuntu, installation on the Inspiron 9300 still doesn’t work out of the box but with a few tricks I was able to install Debian sarge:

  • Boot with expert26
  • When the installer starts up, switch to the second console (Alt-F2) and enter these commands:
    ~ # modprobe ide_generic
    ~ # modprobe ata_piix

    Without this the installer won’t find the CD-ROM.

  • If network configuration via DHCP fails, just retry — worked for me
  • When asked what version of Debian you would like to install, choose stable. Installing testing or unstable directly doesn’t work.
  • It doesn’t matter which kernel you choose to install, we have to replace it with a 2.6.11 kernel later anyway
  • Just before the first reboot, that means right after the installer ejects the CD-ROM, switch back to console two. Now download and install the latest available Debian kernel. I’ve used 2.6.11-1-686:
    ~ # mount -t proc proc /target/proc
    ~ # chroot /target
    sh-2.05b# cd /root
    sh-2.05b# wget http://blog.blackdown.de/static/debian/kernel-image-2.6.11-1-686_2.6.11-7_i386.deb
    sh-2.05b# dpkg -i kernel-image-2.6.11-1-686_2.6.11-7_i386.deb
    …
    sh-2.05b# exit
    ~ # umount /target/proc
  • Reboot (using the kernel just installed) and complete the installation
  • Upgrade to testing or unstable
  • Build a custom kernel (2.6.12 or newer). It’s probably a good idea to include some additional libata patches. To get the DVD drive working you have to apply this patch.

Fixing the ipt_recent Netfilter Module

I have experienced some strange behavior with my ipt_recent netfilter rules after an uptime of about 25 days. The rules started to block much too early. After rebooting the machine I was able to reproduce the problem for five minutes. This clearly indicated a problem with jiffies (Linux initialized jiffies so that the first roll-over happens five minutes after booting).

A closer look at ipt_recent.c revealed that the time tests did not work like intended if one of the last hits was more than LONG_MAX jiffies ago or if the list of last hits contained empty slots and jiffies is greater than LONG_MAX.

To fix this, I replaced jiffies with seconds since ’00:00:00 1970-01-01 UTC’. I have sent the patch to linux-kernel and netfilter-devel. The patch also includes some 64-bit fixes.

May 12th, 2005: The patch has been added to Linux 2.6.12-rc4-mm1

September 8th, 2005: Please note that only the 64-bit parts of my patch have made it into 2.6.12. I’m working on an updated fix for the time comparison problems which will hopefully get accepted for 2.6.14 or later.

September 12th, 2005: These issues have CAN numbers now: CAN-2005-2872 and CAN-2005-2873 (which supersede CAN-2005-2802)

July 10th, 2006: The jiffies issue is fixed in the vanilla kernel now. Also note that 2.6.18 will contain a rewrite of ipt_recent.c.

2.6.12-rc-mm Lockups Fixed

The lockups I have experienced with 2.6.12-rc-mm kernels have finally been fixed by this patch from Oleg Nesterov.

Lockups with 2.6.12-rc-mm on PPC64

After many cycles of compiling and testing ppc64 kernels, I finally found out what causes 2.6.12-rc-mm kernels to lock up my PowerMac G5 regularly: It’s the recent changes in kernel/timer.c.
When I exclude the timer-* patches from the mm series everything works fine again.

I have not found a bug in these patches yet but I am pretty sure that it is no GCC bug.