I have a huge archive of photos. I shoot tens of thousands of photos every year. Storage requirements for all those photos was bad enough when I shot in Jpeg but then I switched to RAW and space usage jumped! Here’s what the last 3 years looks like:
169GB of data is a lot of stuff to store. Originally I had them all duplicated on two external drives but then I bought a 500GB internal drive for my laptop for speedier access. Unfortunately that drive simply wasn’t big enough. I need to convert some of my RAW files to Jpeg to save space. To preserve the original RAW files I want to archive them somewhere permanently. I have a DVD writer so that was an obvious choice.
Burning data to lots of DVDs is tiresome. You can use tar, zip or another archiver to split the data but then you have to run through all the DVDs to pick out a file to restore. I like having the files directly accessible but that means endless selecting files, making sure they’re as close to the DVD size as possible, burning them, moving on to the next bunch. In the bad old DOS days I had a program to fill floppy disks if you pointed it at a directory but I’ve spent years searching for a similar Linux script. Last week I found one.
Enter Discspan. My 2007 archive was already burned to DVD, and I wish I had this script while doing it. I’ve burned my 2008 archive with Discspan and it was a doddle. Point it at the right directory, feed it some details about the DVD drive and let it go. 26 DVDs later and my 2008 archive is safe on DVD!
The script scans the directory, figures out how many DVDs are required and it fills each DVD with data, spanning my digital archive over multiple DVDs.
Be aware when using it that you should let Linux detect the next blank DVD before pressing return. The first time I ran it the script bombed out when growisofs didn’t see media to write to. You also need to patch it because it doesn’t detect the right size of DVD+R’s but it’s a simple one-liner.
Another Linux project, Brasero promises to span disks too but it didn’t for me. It’s the default CD/DVD burner in Ubuntu now and it’s a shame this functionality is broken in it.
Hopefully Brasero will be fixed for the next release. I’d offer to help but my C/C++ is very rusty.
Backed up my entire 2009 archive (so far) to 41 DVDs. That was too easy. I’m happy 🙂
Hey Donncha, why DVDs? the price per GB of HDD is competitive, especially considering that most DVDs sold today are of cheap quality – no good for archival. My experience with DVD backups is that they deteriorate rapidly (had read error after two years on “good” brand DVD), they need to be kept in absolute darkness. I always kept double-copies and checked them (checksums) every six months, replicating on new media when one of the two copies had an error. I stopped. This kind of manual RAID is time consuming, environmentally unfriendly and expensive. I now use offline SATA HDD. On my desktop machine I have a panel with a SATA connector. Plug it in, back it up, and store away in a foam padded pelican case (the main risk is mechanical shock). My photos and videos are currently growing at 10 GB per week. They first go on a 2TB network attached RAID, and once in a while I take the archive HDD out of the case and rsync to it.
Happy New Year
Yuv
I’m not too happy about the idea of archiving to harddisk although for practical reasons it might be the way to go. Buy 2*1TB disks. Duplicate everything and store the drives safely somewhere. I may consider Bluray too. At 25GB per disc it’d take a fraction of the time to archive everything.
A more pressing need is “current” space. Because of duplication for safety I have about 150GB of space on my internal drive. Looks like I’ll have to retire 2007 back to the external drives.
Thanks for the tip! That’s exactly what I’ve been looking for. I also tried Brasero with no luck. Well, that’s not entirely true. I tried to span discs with it and that didn’t work so I compressed the files and then burned them. At some point during the process I had an archive that was too large (I did the math wrong I think) and Brasero offered to span discs for me. No time when I wanted it to span discs has it offered though. I’m not sure what’s going on with that.
As for using HDDs for backup, I agree there. It is more reliable and comparably affordable. However, I do data recovery on a personal scale and a couple of a times a year I’ll need to mail out sets of a dozen or so DVDs at a time. Purchasing and shipping hard drives for data transfers over long distance is considerably more expensive than DVD. Still, I agree when it comes to archiving.
Thanks for the tip. I’m also looking for a solution to do this. And like these comments everyone keeps asking ‘why not use external hdds?’! It’s driving me bonkers! Without going into detail, I want to point a piece of software at a folder and say – ‘Keep burning until all the data is backed up’ without having to use compression, for the reason you specified. Have you come across any other solutions to this? Preferably windows based??
Btw, is Brasero fixed?
Are you kidding? It’s a GNOME project; once it’s broken, it *stays* broken…
Just found Nero DiscSpan – much easier for me to do it in Windows. Hurray!
http://www.nero.com/enu/show-tutorial.php?tut=MTY1
I’ve given up on DVDs now. I use two external drives synced with rsync to duplicate everything. So much handier than DVDs but I really should get a third one to be sure nothing goes wrong..
Having all my data replicated on two disks brings up another question now. What data would I really miss if I lost it? Would I care if the DVD rips I made or my ripped MP3 collection was lost? How’s about my 12 years of photos and videos? I’m running low on space now and these questions haunt me.
At a pinch, the answer is easy. I’ll pick my photos and videos every time!