Backups, “backups” and volatile power

This post is about backups, specifically my backup strategy and it’s latest evolution.  Yes, that thing I suspect most of us working in technology think about, do something half-hearted about and then forget about.

We recently had a few power outages in relatively quick succession (3 in the space of a few hours) which have managed to kill, in “interesting” (read: not immediately apparent) ways, a number of items of hardware in our house.  This included our router (which eventually turned out to be the SSD with the OS on had failed) and the USB hard disk that my backups resided on.  The upshot of this was no copy of the router’s disk or the backups that should have protected from that.

Now, I have not had off-site backups for a very long time, not since my backup was small enough to store on a couple of re-writeable CD.  This was a risk that, until this happened, I accepted.  My thoughts had been a dual hardware failure (taking out both the original and backup) seemed very unlikely and the most plausible situation I thought might result in a total loss (burglary where the original machine and backup device was taken or a fire in our “office” room) was somewhat mitigated by my documents being stored in Dropbox (“other cloud sync-and-share providers are available”).  But total loss of my key machines disk and failure of the drive with the backup has rattled me, so my opinion has swayed to needed a solution which incorporates both off-site and on-site backups.

I had been toying with creating my own backup solution (ugh!), as I didn’t think there was a solution that would fit what I wanted – however I think I’ve come up with something workable using my existing software that meets all my requirements:

  • On-site and off-site copies of the backups (for instant access for restoration)
  • No remote client required – I don’t have root on all the systems I backup
  • At least 1 full copy off-site at all times (i.e. no risk of total loss if a disastrous event occurred while the off-site copy was being updated)
  • Secure (I don’t trust Dropbox sufficiently to put anything I wouldn’t be okay with becoming public in it – my accounts, for example, are held purely within our home)
  • Cost-effective (both in terms of outlay, which largely boils down to space-efficient with the backup data, but also my time to setup and manage it has a value)

I’ve long been a user of BackupPC and I really, really like the product.  Due to it’s file-level de-duplication, it is extremely space-efficient if one is backing up more than a single machine with the same operating system and a lot of thought has gone into making it an efficient and streamlined system (see how it handles compression with a custom ZLib-based format, to minimise memory overhead on inflation, for example) with no remote client to install.  The main reason I had for considering writing my own was struggling to find a solution to the off-site backup solution with BackupPC’s hard-link based de-duplication within the pool (making file-level copying of the pool, with ‘cp’ or ‘rsync’ very expensive processes.

While replacing the failed hardware, I decided to take the plunge and replace my NAS device too, which is a little over 8 years old, that reached the limit of it’s capacity some time ago (at 2TB) and cannot be expanded further.  It’s a little NetGear ReadyNas that has proven itself to be very reliable, so I replaced it with a newer generation with some bigger disks.  The new one supports creating iSCSI targets, as well as the usual myriad of file-based access methods, so I have created a new BackupPC iSCSI LUN which I’ve partitioned and mounted an ext4 filesystem on my Debian box for BackupPC to use.  While musing about this, and the single point of failure I introduce by using the NAS shares for file-storage and the iSCSI Lun for backups on the same device, this solution popped into my head over lunch today:

What I plan to do is convert the plain partition on the iSCSI target to a LVM-logical volume.  I can then snapshot this (the ReadyNas supports manual snapshots of the iSCSI Lun but this appears to have to be initiated by hand through the web-interface, making it hard to script), use ‘dd’ to duplicate the point-in-time snapshot to a USB disk of the same capacity as the Lun for off-site backup without the overhead of cp/rsync having to track all the hard-links in memory.  By rotating 2 disks for the off-site backups I meet my “at least one full copy off-site at any one time” requirement.  Thanks to reducing disk costs and increased density, having 2 USB disks for off-site backup is now doable for under £100 and in a 2.5″ form factor, which wasn’t possible when I setup BackupPC last.


New year or something

As people seem to be posting reflective end of 2017 posts, and I sit on the sofa exhausted but unable to rest it occurs to me how much this feeling sums up the year for me – it doesn’t feel like we’ve stopped all year, we didn’t manage to get a proper break/holiday at all this year but we both did some personally amazing things.  Wendy chucked herself out of a perfectly good plane (and survived) and I ran (yes, seriously I did exercise….willingly!), leapt, swam and hauled myself around a 5k obsticle course.  I’ve changed day-jobs twice, completed my first full year volunteering with a new service and we’re still trying to fine-tune a home routine that works for both of us.  It’s been a relentless, exhausting, exhilarating, eye-opening year and I’m not sure I’d change any of it, given the opportunity.

For 2018 I hope that things proceed a little slower, with more time to rest.  I expect there will be plenty of opportunities to do more, new or personally important challenges.  There’s some important life event planned this coming year, and some time away from work has already been booked afterwards so although we will be at home we should both be able to have a proper break from work and stress.  I bought a treadmill to keep me fit over the winter, which is not being as effective as I had hoped (due to time and too much Christmas food & telly) but I’m still a lot fitter than when I entered 2017 and I hope to keep improving on this through 2018.  My biggest personal goal is to get my weight down to the “healthy” range and be able to run a sub-30-minute 5km.  My OCD is a lot better and I’ll continue to work hard on it, particularly when it tries to take over.

International OCD Awareness Week – 8th to 14th October 2017

This week (8th-14th October 2017) is International OCD Awareness Week, set up by the International OCD Foundation and championed in the UK by the national charities OCD UK and OCD Action.

Information on the week is available on the IOCDF website at
OCD UK’s website is at and OCD Action’s at .

Obsessive Compulsive Disorder (OCD) is a mental health disorder that affects people of all ages and walks of life, and occurs when a person gets caught in a cycle of obsessions and compulsions. Obsessions are unwanted, intrusive thoughts, images or urges that trigger intensely distressing feelings. Compulsions are behaviours an individual engages in to attempt to get rid of the obsessions and/or decrease his or her distress. More information is available on the IOCDF’s website at .

OCD is a much misunderstood and misrepresented condition and the popular image of it being simply a desire for cleanliness, tidiness and/or order is a long way from the reality of this distressing anxiety disorder. In fact, my own misconceptions of OCD based on these ideas made me think my GP was barking when she first suggested that I might be suffering from it!

It’s important to realise that, in the context of OCD, obsessions and compulsions are time consuming and get in the way of important activities the person values. This last part is extremely important to keep in mind as it, in part, determines whether someone has OCD — a psychological disorder — rather than an obsessive personality trait. Obsessions are typically accompanied by intense and uncomfortable feelings such as fear, disgust, doubt, or a feeling that things have to be done in a way that is “just right.” Compulsions are the second part of obsessive compulsive disorder. These are repetitive behaviours or thoughts that a person uses with the intention of neutralizing, counteracting, or making their obsessions go away. People with OCD realize this is only a temporary solution but without a better way to cope they rely on the compulsion as a temporary escape. Compulsions can also include avoiding situations that trigger obsessions.

The IOCDF have a blog post on separating fact from fiction when it comes to what is, and isn’t, OCD:


As some of my friends already know, I was diagnosed with OCD relatively recently. One of the things it has been suggested I should work on, aside from the obvious compulsions themselves, are my feelings of shame. So in that spririt I’m trying to bring myself to publically blog about it a bit.

Most, if not all, people experience times when they go to bed and cannot stop thinking about something(s). They are suffering, at that time, from anxiety – normal, everyday (in the sense of “common” – hopefully not daily!) stress/anxiety that is manifesting as “something on one’s mind”. If that anxiety persists one may decide that, rather than continue to try and forget about it for now (which is probably most people’s first “go to” response to the situation I describe – it’s certainly mine), they need to actually do something about it in order to put it to one side for now and relax enough to sleep. That could be a simple thing, like noting the thing down so you know it’ll be there to remind you in the morning, or checking something is done (e.g. if you can’t shake the feeling you didn’t lock the door before bed) or even spending 20 minutes doing some reasearch on whatever it is so you feel like you’ve done something and can pick it up in the morning. And after this, all being well, you will feel better about whatever it is and go back to bed, feel relaxed and able to allow your mind to drift freely and easily away from whatver it was that was bothering you, hopefully to sleep. This certainly works for me, most of the time, at least.

Now imagine that your mind drifts off and you’re feeling nice and relaxed, and then suddenly after only a few minutes that exact same thing, which was keeping your mind preoccupied earlier, pops back into your head. Now, you’ve already dealt with it, and you tell yourself that, but despite the absolute knowledge that the item in question is done with for tonight you can’t get it out of your head. So you end up, after trying unsuccessfully again to drift off, and again do something to put your mind at rest. Which it does, for a short while. Then it begins again. This is OCD, certain specific thoughts (“The fact these thoughts and their triggers are so specific is what makes this OCD, rather than something else”, to quote the Psychiatrist who made the diagnosis) doing this causing great torment, and this is also the reason I had no sleep last night. It is also bitterly disappointing after a very long spell of having my OCD symptoms largely under control, however life has been extraordinarially stressful for me over the last month or so (somethings that I hope will start resolving themselves in the coming weeks) – so I’m trying to focus my disappointment on the situation rather than myself.

This is the 4th post I’ve written so far, none of the others of which got as far as being published, trying to explain it using a variety of real-life examples (“difficulties”, if you’re a psychiatrist) of mine and analogies. And then today happened, my second (slight, compared to where I was a couple of years ago) relapse this year, and it seemed easier to just write about that.

Plus lack of sleep lowers inhibitions, apparently enough for me to just hit “publish” on this.

About time for another post.

Having written nothing on my blog since the 1st of September I feel it’s about time to flex my inability to spell (made worse by the fact that my sister has stolen“borrowed” the dictionary I keep by my computer) again and write something.

Since my last post:

  • I have quit my job.
  • I have started a new job (“IT Services Specialist”) at Loughborough University, part time.
  • I have returned to my job at Startin Tractors for the other half of the week.
  • my sister has been moved to a more secure secure ward – she’s now locked up with the likes of mentally ill prisoners.

Still it’s all good.

I only wish I had something interesting to put here, but I can not think of anything so instead I’m just going to provide a link to, to annoy anyone trying to work at this point.

…and then there were two (posts)

Having survived another day at work, I’ve now gotten round to writing the final few things I missed off this mornings blog post.

One thing I forgot to mention this morning was that, although MSSQL deleted over 1,000 records from a table by a cascaded delete, the output says “4 rows affected” as only four were deleted from the first table. If a higher number had been reported anywhere in the output it might have allerted to us that there was a problem earlier than the customer calling support because their site no longer functioned correctly.

Rant aside, since my last blog post (in May, this is just an extension of this morning’s) my Grandfather, who was formerly a Commando and then a coal miner, died. He’d been ill for sometime but we did not expect him to die quite so suddenly. Fortunately he died peacfully, in A&E where he’d been taken after coughing up some blood at home.

Yesterday Pete wrote about a document on maintainable code he found at work. The document makes some very good points for writing “maintainable code”. However I would dispute the suggestion that “Every function should be most 20 lines of code”. The rule where I work is that a function should be the length necessary to perform its given task, no more and no less. Usually this means that the function will fall well within the 20 line limit suggested, however it is not uncommon for a complex function which performs a very specific task (such as manipulating the contents of a particular input file, from a manufacturer, to fit the database schema)  to be 100 or more lines in length. Setting a hard and fast limit on the length of a region of code, be it an if block, a function/method, a class, etc. is not, in my opinion, conducive to maintainable code.

Another interesting item I saw noted on Planet Compsoc was this BBC article about Lenovo (who made my wonderful T60) preparing to sell laptops with Linux pre-installed on them. At the bottom of the article it says “Analysts believe that approximately 6% of computers users run Linux, similar to the numbers choosing Apple Macs”. I find this fact extreemly interesting as the company I previously worked for, in the holidays, had a statistics analyiser (which I installed) for their web logs, which showed approximately 6% of visitors to the site used Linux. The Mac quotient of Visitors was significantly less than that, however, and a full 90% of Visitors used Windows XP. Another random fact I found interesting was that use of IE 7 and IE 6 to visit the site was evenly split at 45% each. It makes me wonder how many of those have IE 7 simply because Windows Automatic Updates have installed it for them, and how many of the IE 6 users only have that because they never run the Automatic Updates.

Finally; At christmas I undetook the task of re-writing the stock management system I had previously written for my then employer. The re-write was necessary as the system had started out as a very small and simple thing, which had then had bits and pieces botched onto it as and when my boss decided that it would be nifty to have feature X (or Y or, more commonly, X, Y and Z. By lunchtime.). The result, as always with projects which develop like this, was a hideous mess with, for some reason, worked. Until it stopped working. And then something would hit the fan and land on my desk.

As a result I decided to dump the hacked-to-death php code, and re-write it using an MVC framework. I settled on Rails as it promised great productivity and allowing the developer to concentrate on writing functionality while it worried about the nittity-gritty, such as interfacing with the database. I completely re-wrote a system which had taken over 2 years to develop in 3 months, and Rails did deliver on its promises. Since I’ve stuck to the (somewhat enforced) MVC seperation of the Rails framework adding functionality is a doddle, as is maintaining the code. I have, however, found a small flaw in my approach.

The rails URL scheme opperates on the theme of ‘[controller]/[action]/[id]’, where the controller is the name of the controller (duh!), action is the method within that controller which is being called (and is also the name of the view) and id is an identifier (intended for identifing a db record, for example). I am aware this can be hacked somewhat with the Rails cofiguration, but deviating from the intended path for such frameworks often leads to problems down the line when the framework developers decide to fundamentally change the framework such that these hacks no longer work as intended. Anyway, back to the URL scheme. This is all fine and dandy when I have a stock management system with a ‘browse’ controller, which has such actions as ‘list’, ‘view’, ‘pdflist’ and so on, and an ‘edit’ controller which (also) has a ‘list’, ‘edit’, ‘uploadimages’, ‘uploadpdf’ etc. . (I know it looks like the two list actions violated the DRY (Don’t repeat yourself) philosophy, but they operate in fundamentally different ways, the browse one only operates on a specific subset of the database limited, among other things, to just what is in stock.)

My problem is that, although this is fine for a stock management system, I also need to integrate the old parts management system in as well (on the old system this was a HORRIFIC kludge). There are two obvious solutions, neither of which I’m keen on. One is to create a ‘parts’ controller in the existing app, which contains ‘editlist’, ‘viewlist’, ‘edit’, ‘view’, ‘uploadphotos’ etc. . This could possibly extended to move all of the stock stuff into a ‘stock’ controller. I do not like this as it a) feels too much like bolting the thing on, like the old mess which I’m obviously keen to avoid recreating, and b) the controllers would then get very large and the maintainability provided by seperating out these systems will vanish. The second alternative is to create a seperate rails app to do the parts management. As I mentioned I’m trying to integrate these systems, so creating a seperate app for it seems like a bad move towards that end. It would also mean hacking the Rails config to not assume it is at the root url, and setting up the webserver to rewrite urls. It is all hassle I’d like to avoid.

I’m now wondering if I should have use Django instead, where a project (or site) is supposed to be a collection of apps and I suspect that, as a result, the integrated stock and parts management system would be a lot easier to realise. I’m now back into the realm of trying to justify, either way, another rewrite of the system. I will add that Rails has given me some major performance headaches, and I’ve had to re-write portions of my code to not use the Rails helper functions, which I view as bad, as my code now relies of certain aspects of the Rails framerwork not changing, where as the helper functions should (I would hope) be updated to reflect changes made in the future, in order to achieve something of the order of an acceptable performance.

It’s been a while…

I’ve not posted to my blog since the end of May, so after two-and-a-bit months it’s high time wrote something.

Whilst I’ve not been writing, I’ve also not been checking the comments. Due to the amount of spam, I require all comments to be approved by me before appearing on the site, so appologies to all the people who had comments stuck in moderation.

I’ve now been working in my new job for 2 months and it is generally okay. Windows, VisualStudio (2003) and Sourcesafe are all colluding to slowly drive me insane but for the time being I’m keeping the urge to take a Linux LiveCD into work at bay with healthy doses of Ruby and Debian in the evenings.

The one major cock-up I’ve made at work was a MS-SQL script to delete four rows from a table. Another, related, table had been corrupted and every row had been altered to point to the same (one) record in the first table. I had written a script to delete four faulty record and then fix the data in the associated table. Since I was deleting data I, as I make a point to always do, only used the primary key column of the table I was deleting from to ensure only the specific record which needed deleting was dropped. Unfortunately I was not aware of SQLServers ability to cascade delete record, nor was I aware that this feature was in use of the tabels in question. As a result the related table ended up with nothing in it. Whoops! We are waiting for the backup tape to be sent from Derby to Nottingham in order to restore the data to a point before the script was run. Fortunately all scripts which are run on live database servers have to be peer-reviewed, both for syntactic correctness and that they perform the task intended, before they are run so I have someone to share the blame with. I am, as the script writer, ultimately responsible for this mistake (through my own ignorance) however my colleague who reviewed the script should have been aware of the cascade delete and he did not spot the potential problem either. Nevermind.

For the past week I have also been shadowing another colleague who is left the company yesterday to learn about the systems where he was the only person with any knowledge. Last night hosting services, in their infinite wisdom, decided to move all of the servers involved in these systems from one location to an entirely different part of the country. The one thing that could possibly break everything should have now been performed the very night after the last day of the only person who knew these systems! Go go gadget forward planning.
I have a number of other things to write about, but I have to go to work early today in order to be there should the server move cause any problems. Maybe I’ll find time to write some more tonight (I wouldn’t hold your breath, though).