Migrating from fastcgi to cgi

I’ve just migrated all of the sites on my VMS (including this blog) from running on a fastcgi backend to running on a cgi backend. Why, you may ask. Well, although fastcgi is substantially more responsive than plain old cgi (since it keeps the processes running between requests so the process start-up and take-down times are removed) it consumes much more memory (due to keeping the processes around). Nowadays this is not normally a problem but on a virtual machine with only 128MB available and no swap memory usage becomes a big issue.

By moving from fastcgi to “normal” cgi and tweaking my mysql config I have increased the free memory when the machine is idle from 4MB to 80MB and now have lots of headroom if a single process (e.g. DenyHosts, which is idling at 20MB) decides it wants loads of ram before the Linux low memory reaper starts killing processes.

As a slight aside, the bigest memory hogs on my box are Python programs (DenyHosts [~20M] and rss2email [~45M]). I don’t know it this is a fault with Python or the way the scripts are written (or both, not being intimately familiar with Python I don’t know if it encourages conservative use of memory or not).

Coping when languages change their api between minor versions

I have a rails application(yes, I know, I’ve been regretting it for some time), which I wrote over 2 years ago using the (then) stable rails version of 1.2.3 and whatever ruby version was around at the time (Debian Sarge was the OS of choice for the server). Then Debian Etch was released, so I dist-upgraded to that including the new version of ruby (1.8.5) without any major headaches. Now Lenny is the stable version but the version of rails I used originally does not work with the current version of ruby (1.8.7) because, apparently, `[]’ is no longer a valid method for ‘Enumerable::Enumerator’. This error is thrown in the rails libraries themselves, not my code.

There are two obvious solutions, upgrade the version of rails (which involves re-writing large portions of my code due to api changes in rails, rails is now at v2.3 (http://rubyonrails.org/)) or stick with the old version of rails and install an older version of ruby.

I did the latter. Originally I did this by <cringe>holding back the version of ruby, and its dependencies when doing the dist-upgrade from Etch to Lenny</cringe>. (I appologise for the kittens that were inevitably killed by me doing this!) This did, despite the horribleness(is that a word?) of the method, work.

Today I am installing a new server. Not only am I installing Lenny, which makes manually going to fetch the old versions a pain, but the new box is “amd64″ (it’s an Intel, actually, so x86_64 is more accurate but Debian refers to is as amd64) so I can’t just steal the packages from the cache on the old box. Thankfully all this means that I have been forced to install the old version in some sort of sane manner, by installing Etch in a chroot and calling the old rails app from there. Here’s the steps I took:
(prerequisits: debootstrap dchroot)

# mkdir -p /var/chroot/etch # Make the new chroot directory
# debootstrap --arch amd64 etch /var/chroot/etch http://ftp.uk.debian.org/debian/
# mkdir -p /var/chroot/etch/var/rails # Where the rails app is going to live (well, it'll actually live outside the chroot and be mounted here)
# mkdir -p /var/chroot/etch/XXXXXXX # removed to protect the innocent
# mkdir -p /var/chroot/etch/var/run/mysqld # this will be bound outside the chroot so that rails can access the mysql instance on the server

I added the following to /etc/fstab, and mounted them:

# Etch chroot (for rails)
/proc /var/chroot/etch/proc none rw,bind 0 0
/tmp /var/chroot/etch/tmp none rw,bind 0 0
/dev /var/chroot/etch/dev none rw,bind 0 0
/var/rails /var/chroot/etch/var/rails none rw,bind 0 0
XXXXXXX /var/chroot/etch/XXXXXXX none rw,bind 0 0
/var/run/mysqld /var/chroot/etch/var/run/mysqld none rw,bind 0 0

From here I could enter the chroot and install some needed applications:

# chroot /var/chroot/etch
# apt-get install ruby
# apt-get install rubygems
# apt-get install libfcgi-ruby
# apt-get install rake
# gem install -y -v=1.2.3 rails
# gem install -y pdf-writer

Then I can configure dchroot by adding this to /etc/schroot/schroot.conf:

description=Debian etch (oldstable) for rails

And finally a quick change to the lighttpd config which runs the fcgi program:

"bin-path" => "/var/rails/$app/public/dispatch.fcgi",


"bin-path" => "/usr/bin/dchroot -c etch -d -q /var/rails/$app/public/dispatch.fcgi",

and it all works quite nicely. Now I have a stable Lenny system which I can keep up to date and an etch chroot for the legacy code.

Spot the deliberate mistake…

I just returned from tea to find a computer, which I’d left copying a large amount of data (~300GB worth of backups) from one raid arrray to another, displaying a friendly message:
cp: cannot create directory `./cpool/0': No space left on device.

In order to get to this state I’d done the following:

# mdadm --create -n3 -x1 /dev/md2 /dev/sdc1 /dev/sdd1 /dev/sdf1 missing
(several steps to setup lvm on and format the new device)
# cd /var/lib/backuppc
(some more steps, including adding the new mountpoint to /etc/fstab)
# mount .
# cp -a /mnt/oldbackuppc/pool ./pool
# cp -a /mnt/oldbackuppc/cpool ./cpool

In case you’ve not spotted it, after `mount`ing ‘.’ I needed to `chdir .` to get onto the new mount. As it was I was still on the device that /var is on (~40G) not my nice new 1.8TB raid device!

Kerberised authenticated printing to Windows printers within a 2003 active directory with smbclient

I needed to print to a printer shared via a Windows Server 2003 print server from my GNU/Linux box. Allegedly this should be possible using smbspool, which is provided by samba as a cups back-end to print to such devices. I spent some time looking at it, I was unable to hit upon the right incantation to make it do this. In the end I wrote a short script which uses smbclient and given user’s Kerberos ticket to authenticate, based upon a similar script which used stored credentials to print (to avoid putting username and password in the device URI in CUPS).


# CCLAH 30-March-2009
# Kerberised CUPS printing
# Based upon http://willem.engen.nl/projects/cupssmb/smbc (http://willem.engen.nl/projects/cupssmb/)

if [ "$1" = "" ]; then
# list supported output types
echo 'network smbc "Unknown" "Windows Printer using smbclient"'
exit 0


if [ "$filename" = "" ]; then

# strip protocol from printer
printer=`echo "${DEVICE_URI}" | sed 's/^.*://'`

# Obtain the user's id in order to determine the kerberos cache file name
uid=`id -u $account`

echo "NOTICE: Account: $account uid: $uid" 1>&2

# and print using smbclient
echo "NOTICE: KRB5CCNAME=/tmp/krb5cc_$uid smbclient -k -c \"print ${filename}\" \"${printer}\"" 1>&2

errtxt=`KRB5CCNAME=/tmp/krb5cc_$uid smbclient -k -c "print ${filename}" "${printer}" 2>&1`

echo "NOTICE: Return value: $ret" 1>&2

# Handle errors
# see backend(7) for error codes

# log message
if [ "$ret" = "0" ]; then
echo "$errtxt" | sed 's/^/NOTICE: /' 1>&2
echo "$errtxt" | sed 's/^/ERROR: /' 1>&2

echo "$errtxt" | grep -i 'LOGON_FAILURE' >/dev/null && exit 2
echo "$errtxt" | grep -i 'BAD_NETWORK_NAME' >/dev/null && exit 4

# something went wrong, don't know what -> CUPS_BACKEND_FAILED
[ "$ret" != "0" ] && exit 1

echo "NOTICE: Everything OK"

# success! -> CUPS_BACKEND_OK
exit 0

To use: (at least on my Debian box) save as ‘/usr/lib/cups/backend/smbc’ then add a “Windows Printer using smbclient” type printer with a URI of ‘smbc:/// ‘ and the appropriate driver for the printer at the other end. The only problem is, at the moment, if smbclient fails then the script exits status 1 and the cups print queue enters a stopped state which means that a given (unprivileged) user could theoretically craft a print job which would stop printing working for all users on the local machine.

Setting up a Windows-only network printer for Linux/Mac access.

Due to a lack of a sensible place for this documentation, it’s going on my blog:

We have a Canon MF5770 printer/fax machine which only has Windows drivers. I’m only interested in printing, not faxing, and have the following workaround (which requires a Windows XP machine as a go-between):

Install necessary software:

  1. Install Ghostscript, GSview and RedMon from the Ghostscript site
  2. Install “Print services for Unix” (under “Other Network File and Print Services” in Add/Remove windows components.

Add a Postscript printer to work as a go-between:

  1. Start “add printer” wizard
  2. Choose “local printer” and untick “Automatically detect my plug and play printer”
  3. Choose “Create a new port” and the “Redirected Port” type
  4. Use RPT1: as the port name
  5. Choose a Postscript printer – I choose HP Color Laserjet 4550 PS
  6. Call the printer something without a space (I used “mf5770gs”), and no to default
  7. I shared the printer
  8. No to test page
  9. Open the port settings for RPT1: (printer properties – Ports – Port Setttings)
  10. Redirect this port to program: “C:\Program Files\Ghostgum\gsview\gsprint.exe”
  11. Arguments for this program: “-printer “Canon MF5700 Series” -color -“
  12. Run: Hidden

Print a test page – if all is well it should come out.

I did have to add TCP port 515 to the XP firewall exceptions for it to work, but apart from that the system works flawlessly.

The UK Government strikes again

Since my previous rant on the UK Government’s inability to understand how technology works, it would appear the Government has still not advanced its understanding.

Apparently they are going to force ISPs to record the time, to and from details of all emails. Aside from failing to see how this will possibly prevent terrorism, one has to ask; “what about about those of us who do not use ISPs to send email?” Am I going to be marked as a possible terrorist simply because I run my own mail server, rather than use my [parent’s] ISP’s?

Also, what information are they using as the from and to? If they use the IP address’s of the sender and receiver then they will neither be able to readily find the actual identity of the sender/receiver or to record the IP address of the receiver until they collect their mail from the mail server by pop/imap/webmail. If they, alternatively, record the from/to email addresses then the information will be useless due to how trivial it is to forge from addresses (as anyone who has received spam claiming to be from their bank will be able to testify). If they record the IP of the sender and the email address of the receiver (probably the most sensible combination) then they will still be unable to determine who sent the email easily since a single IP may have many computers behind it (due to NAT routers) and the fact that (especially with ISPs which dynamically allocate IPs) the fact that IP addresses are constantly being re-allocated.

Yet another brilliant, and useless, idea from our illustrious leaders.

Book meme

This seems to be going around t’internet at the moment, so I’m jumping on the band-wagon like the sheep I am:

  • Grab the nearest book.
  • Open it to page 56.
  • Find the fifth sentence.
  • Post the text of the sentence in your journal along with these instructions.
  • Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

My result:

The process’s scheduling priorty, which is a number indicating its importance relative to other processes. (Essential System Administration 3rd Ed, Æleen Frisch, O’Reilly, 2002)

Vista has got to go.

I’ve finally reached the end of my teather with Vista and it has got to go. Why? Not because of UAC, the constant RAM useage of 900MB with nothing else running, the rebooting without prompting me to save whenever windows update feals like it or the fact the a third of my PC games won’t run on it. It’s going because I played two games of minesweeper last night and both times minesweeper crashed (“Minesweeper is not responding”), before I got to the end of the game, in response to me doing nothing more than right-clicking on a square to mark it as a mine (and yes, I did let it send the crash reports off to MS).

On a not entirely unrelated note I spent Monday afternoon slipstreaming SP3 and last Thursday’s emergency hot fix into an OEM XP Pro CD (having discovered I didn’t have an OEM disk at home after taking my boss’s laptop home to reinstall it) using nLite. It was surprising easy and having a CD into which I only have to type the product key and the owner information to get a properly localised British XP install was a very nice experience. One thing I don’t understand is why Microsoft cannot supply localised install disks in the first place. I appreciate that there would be additional cost in producing the different disks but Microsoft is large enough and should be profitable enough to be able to do that. Failing that they could always re-write the installer so that once I tell it I’m in the UK it automatically sets they keyboard, timezone and language preferences (like most Linux distributions) rather that me having to change it in 5 different places.

Oh, incidentally, I am still alive ;) .