The Importance of Backups

How many backups do you have of your important files? I have several backups of my photo archive:

* A local rsnapshot (incremental) backup made every six hours.
* A local copy of my photos directory, which is a straight copy, including deletions. Synced daily.
* A copy on another server synced daily.
* An online cloud backup on Backblaze.

And I still managed to lose a few days worth of photos.

Late last month I took delivery of a new 8TB external drive. I benchmarked it and it was just a bit better than any of my existing drives so I decided to make that the primary location for my photos. I called it “cats”.

Previously my photos lived on a drive imaginatively called “data”, but that would now become my secondary copy.

With my photo archive copied over (all 2.3TB of it), I edited my backup script so it would not accidentally erase my photos on cats, especially not any newly imported photos. I updated rsnapshot so it used cats rather than data, and made sure that Backblaze was backing up cats too. It all looked like it would work perfectly!

The next day I didn’t take any photos, or the next, I took a couple of photos with my phone but didn’t transfer them to my laptop immediately. Then one morning I took a photo of the sunrise, and the following day I did the same.
I edited a few of the images and uploaded one on the same day. Nothing wrong at all.
The next day I went into Cork to photograph Glow Cork and got some nice long exposure photos of the lights. I went in the next day on December 3rd and did the same.

I worked on a couple of shots, but I was busy, and didn’t have time to work on photos until this evening when my world fell apart as Lightroom reported it only had the smart previews for my latest photos. With a sinking feeling and forced calm I checked the 2017 folder. It all looked good, until 2017-11-19 that is. Nothing there after that.

I frantically checked my backup script again and again, poring over each line to make sure the right files were being copied to the right place but it looked ok.
I checked my rsnapshot folder. That should surely have the missing files. No. No, it didn’t.
I checked my other server and the files weren’t there either. They stopped at 2017-11-19.

I checked Backblaze, and the backup made today ended in 2017-11-19 too.

Dumbfounded and a little desperate I started clicking on the end date in Backblaze, going back a few hours, then a day, and another day when I found something weird. I found my December 1st and 2nd photos, but not in the cats drive, but in the data drive!

I’d forgotten to update my Lightroom import preset. Lightroom was still copying my images into the data drive, and then my backup script would delete them the next day. Rsnapshot wasn’t backing them up either because it was now looking at the cats drive.

I started a restore of my December photos off Backblaze. The 3.7GB of data is still downloading. Luckily I hadn’t formatted my camera SD card so my December 3rd photos were still there too, waiting to be copied over again. Once converted to DNG, copied into place, renamed and metadata data updated, Lightroom was happy with those files.

I learned a lot about Backblaze. Without it I would have completely lost my December photos, including this one and the photo at the top of this post.

However, I could not find backups of photos shot from November 20th to 30th. Most of those were sunrise photos, copied to the laptop after the daily backup script had run so Backblaze should have had a day to back those photos up. The restore process offers hourly backups over the last two days, then daily for the next 7 days, then weekly going back a month.

I suspect that Backblaze considered the daily backup done sometime around 7am UTC right after my backup script had deleted the files but that doesn’t explain how I was able to find my December shoot because that was past the hourly deadline. I need to contact Backblaze to find out why.

I still recommend using Backblaze. Last week several factors conspired to make my backup fail:
1. My DSL has a relatively slow 1Mbps upload.
2. I had my laptop unplugged from my external drives for several hours.
3. Backblaze doesn’t immediately back files up.

Then there’s the bad luck to find this problem after more than a week when Backblaze changes to weekly snapshots. It has it’s limitations but it still saved a bunch of my photos and I am very thankful to have it!

I’ve already changed my backup script so it won’t mirror deletions. If I’m going to delete files I’m going to do it soon after copying them onto my laptop and I remove files so rarely it’s not needed in the backup script.

3 months free Backblaze again

A long time ago in 2013 Backblaze had a “sign up and get an extra three months free” offer and they’re back at it again.

For the next 32 days. click on this Backblaze signup link, you’ll get started on a free trial for a month to test it out, but if you decide to pay you’ll get an extra three months.

I love the Backblaze service. It’s a cloud backup service where you can backup your files. I won’t go on and on about it. I did that in my 2013 post, but if it’s any indication of how good they are, I’m still a customer all these years later with over 2TB of data backed up there now.

The signup link is an affiliate link of course (I get three months free as well if someone signs up) but if you don’t want to use it and don’t want that extra three months sign up through this link instead.

And to think, I used to rely on backing up my photos to CD..

Restoring From Physical Media

dancing in the street

Remember DVDs? Last night while looking for the original photo of this dancing in the street photo I was horrified to find an empty directory!

Empty Directories

I had no idea when they were deleted. I searched my backups but of course the directories had been synced a long time ago and they were gone. I checked Backblaze and there was no sign of them. I even checked Google Plus for their backups but still no sign of them. I must have deleted them more than two years ago.

I hate that. I have multiple copies of every photo just because this might happen and yet it wasn’t enough! My backups sync every night so any deletes were synced within twenty four hours.

Almost. Later, I remembered that I had backed up photos to DVDs before I started using multiple drives and Backblaze. They were up in the attic!

DVDs

It didn’t take long to get them, the first DVD worked and my photos from The St. Patrick’s Festival in 2006 were restored, but the photos from April 20th were on the second DVD. The second DVD didn’t work. Nooooo!

The rest of the photos did survive their stay in my cold, damp attic and were restored. I’m thinking now about monthly offline backups to a drive I’ll plug in occasionally.

Do I have any photos from April 20th, 2006? I remember the day well because I was in St. Finbarr’s Hospital with my wife helping a family member. While the others waited I went outside to take photos of a derelict building at the back of the hospital grounds. People passed me going to work while I was take pictures of the daisies, flowers and old architecture. Soon enough a security guard showed up but he just asked me to stop taking photos. I do have a few photos. We called to a friend later in the day and I got some shots of her cat so that’s here too. 🙂

What’s the take home message from all this?

  • You can never have too many backups.
  • You need versioned backups (of a few days at least) in case you notice a mistake a few days later after your backups have been synced.
  • You need an offline backup that’s synced periodically.
  • You need cloud backup that will keep your files safe for a length of time after they’re deleted. Backblaze holds on to your files for thirty days after they’re deleted!
  • When you export files from Lightroom, Photoshop or whatever graphics app you use, always make sure you export a full size original version, not just a web version.

I’m sad that those files are gone, but glad I have a few reminders of an eventful day.

3 months free Backblaze cloud backup!

Argh, I just handed over $95 for 2 years worth of Backblaze cloud backup and now they’re offering 3 months free if you sign up through this link before March 31st! It’s to celebrate World Backup Day, something I’m all in favour of since backups saved the day in 2008 when an external drive died on me.

BTW, both those Backblaze links are affiliate links but I’m a happy customer and I’m currently backing up over 700GB of data to the cloud. 681GB of that is 13 years worth of photos! My upstream bandwidth is horrendous but I still managed to upload 50GB over the last 20 days. At this rate it’ll be a few months before everything is uploaded but the backup hasn’t really impacted on my day-to-day work. Websites and videos still download and display promptly which surprised me. Uploading anything from here usually makes everything else crawl. I told the backup client I wanted faster backups too!

backblaze-performance

It’s not all sunshine and roses though. The client has an exclusion list of directories so it’s easy to exclude directories you don’t want backed up. Sensibly, it doesn’t backup “Program Files” or other system directories by default. However, I’d rather have an include list because on this machine I really only care about my photos, some documents and my Thunderbird mail directory and I know where they live. It’s a small quibble and probably one I’ll soon forgive when my machine goes belly up and I’m desperately looking for a secure cert or the settings for some obscure program!

Backblaze Pod

Curious about where your data lives when it’s in the cloud? That’s a Backblaze Pod there, and it has a raw capacity of 135TB but this post goes into a lot of detail about it and how it’s made. This slightly tongue in cheek post then explains why you don’t want to do this at home!

So that's where all my photos are going
So that’s where all my photos are going

Further on the subject of backups, you should really listen to this episode of The Naked Scientists podcast. This interview with Leo Enticknap, University of Leeds deals with backups but also file formats that scares me. I hope the Canon CR2 raw format is durable enough that it can be read in a few decades, or I may consider converting those files to DNG (which is probably just as likely to be unreadable in the far future TBH).

Try Backblaze, they have a 15 day free trial (or if you’re reading this before March 31st, use this link to get 3 months free) where you can upload data and perform restores to see how well it works. It’s a reasonable price for peace of mind and convenience. My photo archive currently resides in 3 drives on 2 separate computers (using rsync, Samba, Synkron and cronjobs to sync daily) and that won’t change but having an offsite backup like this gives me some confidence in case some local disaster should happen!

So, sorry for the affiliate links but Backblaze is a great service and I hope I’ve made you at least consider duplicating your important files somewhere before it’s too late.