backups Archives - Page 2 of 2

August 16, 2012

Drawing Inspiration and Data Aplenty

Hello everyone! I just came off a bit of a vacation where I went to the Disney D23 Destination D 75 Years of Animated Features event out in California. Of course I didn’t give myself much of a transition – I am finishing up a 3-day private training course today.

Anyway, I’ve talked about it before in a previous blog post, but growing up I wanted to be an animator for Disney. For me, Disney has always been in my life as far back as I can remember; I mean, I can vividly remember coming home in the afternoons and watching the old Mickey Mouse Club reruns on, if memory serves me correctly, Channel 48 in Philadelphia. Disney is liked for many reasons by different people, but for me it’s always been about the animation. So the opportunity to go to an event focused on the animation and while not talked about in that link above, there was a 3rd day that was limited to a certain number of people which allowed you to tour the Walt Disney Studio (but different than the normal D23 tour they do periodically) and see things like the Animation Research Library – no brainer. Going there has always been one of my holy grails. (Before you start rolling your eyes, I promise there will be IT/SQL related content in this post. No, really.)

That third day was simply amazing and worth every penny I paid. In the AM we got to tour the Roy E. Disney Animation building where the animators put together a whole program for us showing the animation process based on the upcoming Wreck It Ralph movie. They took us through the different processes, and not surprisingly, the one that interested me the most was recording the voices and such. Don’t get me wrong, using the real Disney animation tools in a lab to mess with Wreck It Ralph was fun, but I could have sat in that recording studio all day and absorbed info about the process and asked questions since I’m a music/audio guy.

In the afternoon, we got to do different things, one of which was visiting the aforementioned Animation Research Library – arguably the place if you care at all about Disney animation. It’s locked down tighter than Fort Knox. More than a holy grail, it’s proverbially sacred ground and something I thought I’d never see. 65 million pieces of artwork live there. They took a small selection out for us and we got to look inside a few of the vaults. Seeing things like actual pencil drawings from “Steamboat Willie” (1928), the original maquettes used by animators for various movies, the wooden Pinocchio marionette from 1940 seen here (lower right picture), and original Mary Blair artwork (among many other things) was a once-in-a-lifetime experience. Most Disney people can’t even get in, let alone see some of what we saw. I am still thinking about it days later.

Another standout thing was that we got various presentations, one of which was on the preservation and restoration of the Disney vault of films (animated and live action) and shorts. This was something right up my alley since I’m also a video guy (I still have and use my laserdisc player). It was clear I was the techie guy asking the questions and people were looking at me like I had two heads. It was definitely a magical day all around.

As much as I was on vacation and loving this experience, my IT/SQL mind started to creep in and come up to the foreground every now and then. When we were in the studio, some folks go to do some ADR work to see what that process was (and it was fun to observe), but I was a bit more interested in the gear and process. What was interesting to me is that besides the takes they recorded in one rig, they had a continuous session running in parallel on a different one to capture literally everything. One reason for that is simple: if something is missed, it is captured elsewhere (think backups and HA/DR – gotta love redundancy).

That begs the question: how much data is that? Well, potentially a lot for each project over its lifetime. So of course I asked the question – at what resolution do they record at? For you non-audio geeks, to put anything I’m going to say in perspective, a CD’s resolution has a sampling rate of 44.1 kHz at a depth of 16-bit (an 80 minute CD at 44/16 is roughly 700MB of data). In the audio world, higher sampling rates (48, 88.2, 96, 192, or 384 being the other common options) and more depth (either 24- or 32-bit) is generally a better thing. Not surprisingly, that all chews up a lot of space the more resolution and the higher the bit depth you go. They record at 48/24, and everything is stored on a server with a lot of space. Considering how much audio they must have for each project, that’s a lot to deal with (even at 48/24) not only in terms of needing it near term for the active movie project, but then to back it up and ultimately archive it for other uses down the road. This is not unlike a DBA needing to manage a total of giga-, tera-, or petabytes of information both in databases as well as the various copies (including backups). This is just the audio data – can you imagine all of the animation, too? Wow. Oh, and every bit of that audio needs to be meticulously cataloged, too. Much like you need to keep inventory of all of your servers, instances, and databases.

During the video restoration presentation, we (royal we here) got to talking shop a bit (probably to the dismay of the non-audio and video folks who were in my group) and I asked why they only scan in at 4K and not 8K, as well as discussed audio sample rates. The reason was similar: it’s a data problem and for some old film types, you’re theoretically not going to really get better with higher scans (MuppetVision 3D in Disneyland was scanned in at 6K if you care about such things, but that decision had a lot to do with the original format; this forum discusses this topic more if you want to see some opinions and other links). This puts a whole new spin on Big Data, eh? It may not be schemas, but it sure is a ton of information that is unwieldy to deal with.

What was also interesting about that presentation was the talk about the film stock and not only how flammable it could be (depending on its type), but no matter what they do, at some point, the originals will be completely unusable (much like how I related backups to audio which I blogged about awhile back). Digital, film, analog tape – stored properly it will have a long shelf life, but it may not … it all depends since there are so many factors that contribute to that. If you have important data that needs to live for a long time, preserve it properly and have adequate redundancy, too. Archiving is no joke. A company like Disney is built on its history, and they need to preserve it.

Shifting gears a bit, the Animation Research Library as I said houses 65,000,000 pieces of art. I didn’t ask, but I could tell it was all cataloged and scanned in (I thought I heard someone say 150 dpi which was good enough years ago; I believe they’re working on scanning in at either 600 or 1200dpi now that tech has moved on and storage has gotten cheaper). Anyway, point is, we’ve got another data thing going on here. This is data not only used by animators today that needs to be accessible (hello HA!), but quickly and organized. Side note: cool perk of working there is you can apparently choose pieces from the archive to put up in your office. Drool. This is Art on Demand on steroids! Where do I sign up at HR for framed pencil drawings of “The Band Concert“?

Would I still love to be an animator? Maybe, but some of what goes on at Disney isn’t too far off from what I do today. I help customers ensure their data is available when they need it, be it near line or offline. Your data is your IP … and legacy. So maybe I really did choose the right career path after all – just getting a glimpse of how Disney deals with things like archiving and redundancy gives me inspiration in what I do right now, and is more than just a doodle on a page or a ride in an amusement park.

September 11, 2011

Are You Ready?

It’s hard not to be a bit reflective today – 9/11. The images are just as disturbing today as they were 10 years ago. I remember exactly where I was. I had just barely opened my eyes when I saw the second plane hit. The next 100 or so minutes are seared in my memory forever. My parents grew up in New York (the Bronx to be exact), and my maternal grandfather worked in the World Trade Center for the Port Authority before he retired. So I have a spiritual connection to New York. It’s a city I do love to visit (but those hotel room costs at certain times of the year … oy!). I remember writing the initial chapters to my SQL Server 2000 High Availability book and originally I had a 9/11 reference in it. To a person, everyone reviewing it said to take it out because the reference was too soon. And I did.

I also remember driving down to NJ about a week after 9/11 for Rosh Hashanah (the Jewish New Year). It was at night (probably 10 or 11 o’clock). When I passed the commuter parking lot at the top of the Garden State Parkway attached to the Montvale Rest Stop, I saw a lot of cars there. That was unusual. And I realized – a lot of those cars probably belonged to some of the victims of the World Trade Center. It was eerie, and a feeling I never want to have again. I can’t describe it. It’s funny what sticks to you years later.

Over the past 10 years we’ve seen a lot of natural destruction around the world – the tsunami from the Indian Ocean (which is sneaking up on its 10th as well), the recent earthquakes and tsunami in Japan this year, Hurricane Irene just a few weeks ago which has caused massive problems here in the Northeast, and Hurricane Katrina. Those are just examples – I know there are many more. Could your business stand a natural disaster that hits your area? Have you even taken it into count in your overall operations plans? If the answer is no to either, you may be in trouble. It’s not just natural events that will get you. The recent power outage in Southern California, Mexico, and Arizona is a great example of a man-made problem which can have a very adverse effect on your business if you don’t plan for those kinds of things.

At the end of the day, when the dust has settled, life usually tries to return to (relative) normal. Businesses reopen. Workers go back to work. Money is made. People get a paycheck. In this connected world, it’s not as simple as “opening the doors”. Everyone’s got e-mail systems, database systems, application servers, SANs, networking, communications … you get the idea. I talk about this with customers, but the reality is that everyone needs a disaster recovery (DR), or more accurately, a business continuity (BC) plan. BC is more accurate since there are more elements (such as human ones) that come into play.

I’m not saying you should ignore the technical side – far from it. Plan to your heart’s content. Do things like get your backups offsite. Test your DR plans. Make sure you know how to coordinate resources. Have a staff schedule. You get the idea. The trust is that all of those tactical – and necessary – tasks are very important stuff. It’s mainly what we focus on as technical folks. However …

What brought this more into focus for me recently was the death of my friend Mike which I’ve blogged about here. Besides the loved ones he left behind, he also left behind a law practice and a computer consulting business with active clients that still needed to be serviced. He may not have been Accenture or IBM Global Services, but his business still has to run until the family figures out how to transition the work properly and legally to others. From my perspective, it really got me thinking. It’s one thing to prepare for certain things, and I’ve talked about people being single points of failure in the past, but would happen if something happened to me? What would become of my business? How would those I entrusted in my will to my estate deal with things? Would they know where to find my contracts? Do they know my passwords? How would they see my receivables? Well, you get the picture. Those are very different concerns than just worrying about getting an application running – but they are every bit as valid. So whether you are a 1 person company or a 100,000 person company, these are the types of things that come with BC.

I’ve started to re-examine my own contingency plans. Have you? If not, get going!

10 Years – What A Difference

When all of the things happened around 9/11 or other events like the tsunami in the Indian Ocean, we saw the many pictures posted all around those areas of people looking for loved ones with contact information on the flyers or posters. Facebook and Twitter were not around. I believe listservs were used by the Red Cross to try to track the missing. I do remember rudimentary web pages. Today, there would be Facebook pages and real time tweets. My, how far we’ve come. I’ll be the first to admit I begrudgingly embrace social media in some aspects, but when it comes to things like this? It’s very, very good.

On Another Note

9/11 has obviously changed the way we all go about our lives whether directly or indirectly. I fly quite often, and it has been interesting to see the changes in airport security all over the world. I can remember flying a few months after 9/11 and that experience was WAY different than what we have today. It was certainly a sight to see armed military walking around. In those days immediately post-9/11, some of you may remember not being able to take ANYTHING with a battery out during flight. No laptops, no audio players, no phones – nothing. Laptops in bags had to go in the overhead. You were stuck with your travel companion (if you were with someone) and things like books/magazines/puzzles. At that point you could still take your own water (this is pre-Richard Reid). As much as I do like to work on planes sometimes, I’ve got to say, outside of boarding planes in Europe, it was the fastest boarding process I’ve ever experienced. I kinda miss it. In the USA, nowadays no matter what airline, it’s a bit chaotic with people cramming things in overheads to avoid checked baggage fees and thinking they are so self-important that they can ignore flight attendants when they are asked to power down their phone or smartphone. Hint: you’re not special. That e-mail or conversation can wait. You’re holding us all up.

I’ve also noticed a slight change in some folks (like my parents) when I go somewhere. I’ve criss-crossed the globe usually alone. My Mom doesn’t fly, and I know she gets anxious when I fly. I definitely appreciate the concern (nice to know people care!), but the reality is that if something happens, it’s going to happen. I can’t live my life in fear of “what if”. I certainly know my friend Mike didn’t. Sadly, I probably have a better chance of something happening to me driving or walking around a major city (not that I want anything to happen) than something happening on a plane. The numbers back me up on that. I can only control my actions – not anyone else’s.

Life is certainly fragile, but don’t put yourself in a cocoon. Live it to the best that you can. You’ll regret it otherwise.

June 7, 2011

Dumb SQL Questions

Meme Monday, started by Thomas LaRock (Twitter | Blog), is an interesting and fun thing. This month, the topic is dumb SQL Server questions. There are a ton, but this week I’m going to focus on three I hear all the time.

Question 1: My Windows/backup guys make a full backup of the system with SQL Server. Isn’t that the same as making a SQL Server backup of the databases?
In most cases, the answer is no. SQL Server is NOT a flat file-based database. While a Windows-level backup of a SQL Server database can be done properly if it interfaces with VDI/VSS, chances are most backup programs won’t handle that or may skip the SQL files since they are in use. Plus, at that point, you would most likely have to attach files – it’s probably not a backup you could use in the traditional RESTORE sense or use transaction log backups with. Some software may do proper SQL Server backups, but you have to ask. Some hardware-based solutions do properly interface with SQL Server, so while out of control of the DBA, can make valid backups that are logged in SQL Server. You would need to check with your vendor.

At the end of the day, as a DBA, you’re responsible for your SLAs, RTOs, and RPOs. A big part of that is ensuring the right backup strategy. Leaving it to your Windows or backup guys could be a potenital excellent career limiting move.

Question 2: Isn’t SQL Server failvoer clustering the same as Oracle RAC?
No, no, a thousand times no. While maybe not a dumb question, it’s the one I hear the most in clustering discussions/talks and it’s clear people don’t do their homework and are making some pretty big assumptions. SQL Server’s failover clustering is built on top of Windows failover clustering. A single server (node) can own a given clustered instance’s resources at a single time. Shared disk is the single point of failure. Scale up is how you would get scalability.

RAC allows you to scale processing, but you still have one copy of the data on disk – single point of failure, and if you have an underpowered storage solution, I don’t care how many processing servers you have – you’re not going to scale if your I/O sucks. Plus, while I know it works, the lock manager Oracle has coded is a complex piece of technology.

Both solutions are not going to necessarily be straightforward to implement. Microsoft has made clustering a lot easier over the years.

At the end of the day – both are valid approaches and are proven technologies.

Question 3: Can’t SQL Server have zero downtime and no data loss?
To achieve zero downtime, if you never patch SQL Server or any other components (including Windows) and never have anything unexpected happen, sure – you can have no interruption in service. Outside of that, no way. Someone recently tweeted they need a maximum of 8 seconds of downtime. That is darn near, if not, impossible. You have to be realistic about your tolerances and what is actually achievable. With the best processes and technologies, you’re looking at anywhere from 30 seconds to a few minutes in a best case scenario.

As for data loss, it’s going to be an “it depends”. If you’re using some kind of synchronous method (i.e. high safety database mirroring), you won’t have data loss for that database since it must be on the mirror as well. But there are complexities – how in your application can you tell where you are at data wise? What happens if your application uses multiple databases – how will you keep them in sync and then tell where they are at data wise?

Failover clustering will always be consistent to the point of failover because the stop and the start means that anything that is incomplete won’t be written will be rolled back due to the recovery process, but you need to account for figuring this out in your application.

December 31, 2010

No Backups, Big Problem

I figured I’d end the year with a big bang of a post.

Whenever I talk about high availability or disaster recovery, I spend a bit of time on backup strategies (among other things). People sometimes give me a weird look like, “You should be talking to me about clustering, mirroring, and everything else, Mr. Clustering MVP. Let’s go!”. The reality is without backups, you have nothing. My blanket piece of advice is always this: if you do one thing right, do your backups well.

I know of customers (some who are even larger) who are still not doing things like:

Copying backups to another location after they are made
Testing the procedures to find and acquire the backups (especially if they were multi-tape or disc sets)
Testing a restore of the backup
Applying proper organizational techniques

Why are these important? Well, it’s simple. I’m going to use music to tie it back in to SQL Server because the music industry has some interesting parallels. As some of you may know, I’m also a musician. Growing up largely in the digital age, most of the things I’ve done since 1994 have been in the digital domain when I purchased my first DAT (digital audio tape) deck post-college as a graduation present to myself (a Denon DTR-80P). I still have it and it works well. My second DAT deck (a dual well Tascam) was a piece of junk compared to it – and it was a piece of studio gear!

DAT in terms of a music format was killed for copyright reasons in the consumer market, but flourished in the pro/semi-pro market. It’s a direct descendent of things like the Sony PCM-F1, which allowed you to record PCM digital audio direct to Betamax machines. My first All South Jersey Junior High jazz concert was recorded by one of the conductors with this combo. A part of me wants a F1 for the heck of it, because it’s just so cool.

For those of you who care about such things, DAT is a 2-track (i.e. stereo medium). ADAT (Alesis) and DA-88 (Tascam) were the affordable digital multi-track formats for pro recording in the 90s for many digital-based music projects as hard drives were not large or cheap enough yet. There were digital multitrack machines going back to the late 70s/early 80s from Mitsubishi (such as the X80, which was used to record Donald Fagen’s classic The Nightfly, but had an odd sample rate of 50 kHz in early machines), 3M, and the Sony DASH machines.

Laserdiscs came along in the 70s, and were the precursor to the compact disc we all know … and don’t universally love. The CD gave way to DVD, and now Blu-Ray (let’s forget about HD-DVD). In the 30 odd years digital has been a “mainstream” format, we have been promised that digital was supposed to last forever with no problems. Right? Wrong.

Rolling Stone just published an interesting article entitled “File Not Found: The Record Industry’s Digital Storage Crisis” which highlights a big part of the problem (but not all of it). In this digital-based music world, where anything physical is now looked upon as “outdated”, there are big issues. I’m not going to get into a sound quality debate nor am I going to lecture on the importance of legally acquiring music, as that’s outside the scope of the goal of this post. I think you can tell where I probably sit on those issues, anyway.

Where’s The Backup?
Obviously the title is a riff on the old Wendys commercial here in the USA, but it is appropriate. One of the biggest issues the music industry has faced in the past 10 – 20 years is finding the original tapes (mono, stereo, or multitrack) so things like remastering, archiving, surround sound projects, etc. could happen.These tapes are the collective IP, or data, of each of these companies no matter how big or small they are. No tapes, no revenue and bringing things into future media formats.

You may have heard tales where artists, labels, or studios threw out or destroyed tapes years ago just to make room somewhere. Well, a lot of that is true – just read the bit here on the Who song “I Don’t Even Know Myself”. A few years back, Steely Dan didn’t pursue a surround SACD for their seminal album Aja because they couldn’t find the complete set of multitrack masters. We’re not talking about some obscure a non-hit album with cult status; we’re talking one of the most popular and well received albums (arguably) of the past 30+ years by critics and fans alike.

Another problem is that a lot of the music industry is now using a third party like Iron Mountain to store their precious masters and they are not kept onsite. If the tape is labeled wrong, or is just stored without any thought or oversight from the record companies, finding a tape (whether it’s the stereo or a multi-track master) will be like a needle in a haystack. There are famous stories about master tapes being found labeled with words like “do not use”, and safety copies have been used for all of these years for subsequent releases.

Something more of you may relate to is your music collection that you may have culled for your portable player such as an iPod. Whether you are using iTunes or just storing the music files on a hard drive somewhere, many people have amassed large collections which amount to terabytes of music. For those who ripped the music from their own discs and didn’t purchase the music from online stores, you have invested a lot of time. Some of those folks even assume their files are safe, so they dump their physical media and just keep the digital file. Do a quick search using your favorite internet search engine and you’ll see that there are many stories recounted in forum posts, blogs, etc., about how someone’s music player or hard drive died, and they lost their entire collection. It’s not uncommon to now hear people making multiple backups of their files

Now apply all of this to your own world of SQL Server and unerlying Windows systems:

Make backups frequently that match your stated recovery point objectives (RPOs) and recovery time objectives (RTOs). Can you afford <insert number of hours/days/months> of data loss? If the answer is no and you are making backups less frequently than that, you’ve got a problem. One size does not fit all. Someone may not think your system is mission critical, but if your business will come to a standstill if it’s down, well, do the math on that. Time to make up some new rules, boys and girls.When it comes to system and system state backups, depending on your processes and other factors, sometimes it is easier to just do a bare metal install. These days a base Windows Server 2008/2008 R2 installation is VERY quick. This does not mean you should never back them up or the critical files, but do what is right for your environment.
If you are doing backups to disk for your SQL Server databases using native tools or third party ones from companies like Redgate, Quest, etc., are the resulting backup files being copied elsewhere? If not, you have a single point of failure. It’s great that you made the backup, but what happens if the drive it sits on dies? No more backup. How does that work for your RTOs and RPOs?
In most companies, the process for copying the file and archiving it usually the responsibility of another group in IT outside of the DBAs and the Windows folks. Do you know the process for getting that backup? Have you tested it? At the end of the day, it’s your job – not the backup operator – to get the database or system back up and running. If you have not done anything, including having at least a working relationship with the backup operators, you might as well have a pair of dice around to roll when it comes time to get an older restore for whatever reason. If it is due to a major outage, you may even want to have a resume handy. I’m not kidding.This is a bigger epidemic in environments that use network-based backup programs and no local backups are done at all. Things are really out of a DBAs hands there. Good luck to you if you’re in that situation, have a 1TB database, and you’ve got a 15 minute RPO.
Forget the process for getting the backup – do you even know where it is? Do you keep track of where you store them? Are the locations standardized? Are the names of the backups standardized, and do they include things like dates? I mean, you’ll never do any kind of restore if you can’t find the backup files. You have to know what you’ve got to make critical decisions. I talk about things like this in my older 2005 HA book which was not just a clustering book. My new Denali-based book will have a lot of these types of discussions as well …
Let’s assume you’ve got backups and you know where they are. Do you test the backups? This is arguably my biggest pet peeve with nearly every customer. Making a backup just generates a backup. It does not mean it is good. The only way to test a backup is to do a restore of it. That may be impractical to do every night, and especially if you’ve got thousands of DBs and/or larger ones, nearly impossible, but for your mission critical systems with low RPOs and RTOs, it’s suicide not to test them at least periodically. Plus, you can answer the age old question from your C-level, “So how long will the restore take?” when you’re in that scenario.
Do you ensure that your backups are not a single point of failure? As I mentioned earlier, some people generate their backups and they live in one place, and at some point, older ones are deleted to create the new ones. That’s all well and good, but if these backups are not being copied elsewhere, you lose where those backups are or the drive fails, you’ve got nothing. It’s your job to ensure that not only are you protected at the instance and database level with things like clustering and mirroring, but your database backups are just as redundant to a degree.
Archiving backups is a big issue for some that are regulated to have them for like 10 years. That’s a lot of space, whether it be tape, disk, or some optical medium. Along the same lines, have you planned properly for your current backup storage needs? I find it’s often overlooked in system planning.

Forensic Science
Rolling Stone’s tagline says, “Vinyl and analog tapes last forever, but hard drives fail and digital formats change”. That’s a bit of a fallacy in the music world. Anyone who has worked with analog tape knows that at some point, it can become unplayable. Analog tapes can shed oxide (which essentially holds its “data”). Analog tapes can become moldy if stored improperly, and may not be able to be cleaned (it’s noxious stuff, too). There is a process called baking which will allow the oxide to re-bond with the tape material so it can be played again, but it may be its last play. The point is to get it so that it can be archived it in some other format (digital or analog). That’s the reality of analog. Forever? Not in my world.

Even standard audio cassettes (you remember those, right?) have issues over time: they become brittle, suffer from dropouts, etc. The cassette was a convenient format (much like MP3 has become the new de-facto standard for portable music, much to the chagrin of some), but hardly perfect. It was good consumer format that had a good life.

Digital audio on a tape-based medium has problems as well. For example, it can (and often does) suffer from digital rot. Basically it is when the tape deteriorates (for whatever reason), loses the information and if it is playable, you hear what is akin to white noise. This is somewhat like drops in an analog tape, except because it is 0s and 1s, the information isn’t coming back. This has happened to me; one of my first recording projects was lost to the sands of time because of this. I only discovered it when I tried to digitize it and burn a CD-R in the late ’90s … and we’re talking about a recording session from maybe 5 or so years before.

Two of the jazz albums I recorded between 1997 and 2001 (which will soon be available for sale again, but only as digital downloads; you can contact me or post a comment if you want more info) were recorded on ADAT. In 2005/6, I wanted to remix them and archive the masters. Well, that was easier said than done. I had the original ADATs, but playing them was another story. Some were problematic, but luckily they were playable at least once. There was a very real chance I was not going to be able to do what I needed to do.

This also brings out another problem: formats, playback equipment, and software programs and versions. A lot of the music (as well as video) industry records on formats popular at the time, but later may no longer be in use. The music industry is lacking in overall standards (although some are prevalent, such as MIDI and SMPTE for synchronizing tapes as well as music to video), so that complicates things.

The first thing you need to worry about is having the right hardware to play back the tape, and ensuring it is in good repair. Older formats may no longer be in use anywhere, so locating a machine, let alone parts, can be a tricky AND expensive endeavor. Even ubiquitous formats such as ADAT – only out of favor for a few years – are getting harder to deal with because everybody dumped their machines and jumped into computers and hard drives.

Then we come to software. Many of the modern programs we use today such as Cubase have their roots in the 80s on computers like the old Atari ST. I was an Amiga man myself, but you get the point. Even Macs which have been used in the creative world for a long time have gone through multiple iterations of processors and configurations, so a piece of software designed, say, to work on an old PowerMac won’t necessarily work on a newer Intel one if at all. And the older software may not even be backwards compatible, so files created on an older version, even if salvaged, may not be able to be opened. Add to that the use of things like software-based plugins instead of hardware processors for audio, and that complicates things even more because the plugins may only work with a particular version of its host. This also assumes your tapes/disks/hard drives are still playable, too. Are we having fun yet?

So let’s assume for a moment you’ve solved the format, hardware, and/or software problems, especially for more modern records which have been recorded on computers with things like the infamous ProTools. Do you have extensive session notes about the mixes and which bits of audio are in use? Do you have all of the files needed? Is everything logged and labeled properly? Chances are the answer is a resounding NO since no one thought of that while tracking, mixing, or mastering. Trying to do something with those files later will be a bit like alphabet soup.

This has come to the forefront with games like Rock Band and Guitar Hero where bands are giving these companies their multis to put into the game, but they are either missing master tapes (old analog or digital) or certain audio files (hard drive) are gone and lost to the sands of time. So the bands either do full re-records, or just re-record the missing bits.

A great article on all of this is “Remixing Depeche Mode In Surround” from my favorite magazine, Sound on Sound. Particularly look at the section “Archives & Pre-production”. It touches on nearly every aspect I mention in this blog post.

So what does this mean for you in the non-music, IT world?

Media dies. Good example: USB keys. They’re convenient, but have a shelf life. Another example: hard drives. Heads can become frozen. Yet another example: tapes of any kind. We’ve all had one unfurl on us. You get the idea. Don’t assume that the medium you are using now will still be playable in 5 years. Test reading from them, especially for milestone backups you need to have around for whatever reason.
Keep accurate documentation about your backup sets and what is needed to recover your systems or databases completely. I have some scripts which you can download right here on this site from my 2005 book which will generate the RESTORE command for you based on the types of backups you used and what is in msdb. Someone else who had no involvement with the backups may be called on to restore them, so having detailed information is critical.This will also help identify which tapes and disks (or location in a vault) are needed in addition to the files themselves, and where you can locate them. All of this matters in a server down situation where you need to restore from bare metal. If you’re missing stuff, well, you may be out of luck and a resume may be nice to have.
Like the music industry, there has been no shortage of hardware, backup formats, and software. Do you have the right stuff to restore that 6 year old tape (assuming it’s playable)? Probably not. That tape drive was probably sitting in some storage closet a few years ago and decommissioned, and most likely given away or thrown out at some point. eBay may be your friend, but it won’t always yield what you need. Software may be the hardest part of that equation, as hardware can usually be found to make the restore. You may have a later version, of say, NetBackup, but it may not be backwards compatible with the format of the backup on the tape you’ve got. You may not even have that old version lying around. At that point, chances are you are out of luck. I’m not trying to be harsh, but that’s reality.

So as you can tell, backup and restore is a serious matter, and much more complicated than it appears to be. You need to be vigilant, especially if it’s your butt on the line.

Categories