Allan Hirt, Author at SQLHA

August 22, 2012

Pick Any Two

It’s been said that you can do things right, cheap, or fast. Pick two.

I’ve heard people substitute other words into that first sentence (such as available, reliable, and fast), but you get the point: you generally won’t get it all when it comes to deploying a solution or working on a project … that is, unless you’re willing to pony up some serious money. Even then, it still may not be possible due to some other limitation. Truth be told, of that initial list, I might argue you only get one of three (right and fast are not always friendly bedmates), and even that may be a question mark. Why? This all underscores an important point in what we do in IT as well as in life: often times, it’s about tradeoffs. The right solution or decision is often the most difficult one to make or implement. No one likes to work anymore, right? 🙂

Whenever I see offerings from various companies offering you solutions that promise you the world (unlimited scalability, five nines of uptime, we manage it for you – nothing for you to worry about, etc.), I just chuckle. Either they did the right work on the backend to make that happen (not likely) or they’re selling you a bill of goods (maybe). The truth probably lies somewhere in between.

It takes effort, knowledge, skill, budget, and often, a bit of luck for the stars to align. How many times have you joined a new company with the illusion you’ll change the world or started a project completely idealistic, only to find things like a small budget spoils the plans and you’ve got to figure out how to “make it work”? This is also why in many cases, besides them not possibly applying to your scenario, best practices are often hard to implement. They’re not always real world. “But I must have RAID 10!!!!” Keep on living the dream when your SAN guy or gal gives you whatever they have and you have no choice in the matter.

It’s nice to be idealistic. Maybe you can join Daryl Hall in Dreamtime (and as evidenced in this clip, he still looks/sounds pretty damn good in 2012 at 65!)

http://www.youtube.com/watch?v=Re805lUUfvU

Here’s my reality: I live in the real world with customers that have resource, time, budget, and skill set issues (among many other constraints; every customer is different) which need to be taken into account when I deliver something. It’s not about being right (in some way it is, but standing on ceremony gets you nowhere) or saying “I told you so”. At the end of the day, you have to be realistic, and so does your management as well as the bean counters. You’re not getting 24x7x365 and great performance on things like limited budget, staff, and severe time limitations (a subject I addressed a little here). Something has to give. Sometimes it is a technical reason that something isn’t possible. Yes, we’ve come very far in the past 10 – 20 years, but everything isn’t possible. And Microsoft isn’t changing SQL Server or Windows just for you to make you work a certain way, either. Even if you offered to fund that development, I doubt it’d happen.

The worst offenders here I find are managers or C-levels. It’s not all of them, but many. They may have once been technical, but now they may be removed from the day-to-day of what it takes to get something done. I mean, don’t we worker-bee types all have magic fairy dust to create a /faster or /moreuptime switch? If I did, do you think I’d still be working? I’d be selling and/or licensing the hell out of it and retired.

What you need to do, and I do it all the time, is get everyone on the same page with the right information so you can move forward. Like I said, some things will have to give. That may mean you need more budget, time, etc., or hard cuts will have to be made to make other things happen. That’s life. When we can have the right conversation without anyone digging their heels in, the real work can begin.

August 16, 2012

Drawing Inspiration and Data Aplenty

Hello everyone! I just came off a bit of a vacation where I went to the Disney D23 Destination D 75 Years of Animated Features event out in California. Of course I didn’t give myself much of a transition – I am finishing up a 3-day private training course today.

Anyway, I’ve talked about it before in a previous blog post, but growing up I wanted to be an animator for Disney. For me, Disney has always been in my life as far back as I can remember; I mean, I can vividly remember coming home in the afternoons and watching the old Mickey Mouse Club reruns on, if memory serves me correctly, Channel 48 in Philadelphia. Disney is liked for many reasons by different people, but for me it’s always been about the animation. So the opportunity to go to an event focused on the animation and while not talked about in that link above, there was a 3rd day that was limited to a certain number of people which allowed you to tour the Walt Disney Studio (but different than the normal D23 tour they do periodically) and see things like the Animation Research Library – no brainer. Going there has always been one of my holy grails. (Before you start rolling your eyes, I promise there will be IT/SQL related content in this post. No, really.)

That third day was simply amazing and worth every penny I paid. In the AM we got to tour the Roy E. Disney Animation building where the animators put together a whole program for us showing the animation process based on the upcoming Wreck It Ralph movie. They took us through the different processes, and not surprisingly, the one that interested me the most was recording the voices and such. Don’t get me wrong, using the real Disney animation tools in a lab to mess with Wreck It Ralph was fun, but I could have sat in that recording studio all day and absorbed info about the process and asked questions since I’m a music/audio guy.

In the afternoon, we got to do different things, one of which was visiting the aforementioned Animation Research Library – arguably the place if you care at all about Disney animation. It’s locked down tighter than Fort Knox. More than a holy grail, it’s proverbially sacred ground and something I thought I’d never see. 65 million pieces of artwork live there. They took a small selection out for us and we got to look inside a few of the vaults. Seeing things like actual pencil drawings from “Steamboat Willie” (1928), the original maquettes used by animators for various movies, the wooden Pinocchio marionette from 1940 seen here (lower right picture), and original Mary Blair artwork (among many other things) was a once-in-a-lifetime experience. Most Disney people can’t even get in, let alone see some of what we saw. I am still thinking about it days later.

Another standout thing was that we got various presentations, one of which was on the preservation and restoration of the Disney vault of films (animated and live action) and shorts. This was something right up my alley since I’m also a video guy (I still have and use my laserdisc player). It was clear I was the techie guy asking the questions and people were looking at me like I had two heads. It was definitely a magical day all around.

As much as I was on vacation and loving this experience, my IT/SQL mind started to creep in and come up to the foreground every now and then. When we were in the studio, some folks go to do some ADR work to see what that process was (and it was fun to observe), but I was a bit more interested in the gear and process. What was interesting to me is that besides the takes they recorded in one rig, they had a continuous session running in parallel on a different one to capture literally everything. One reason for that is simple: if something is missed, it is captured elsewhere (think backups and HA/DR – gotta love redundancy).

That begs the question: how much data is that? Well, potentially a lot for each project over its lifetime. So of course I asked the question – at what resolution do they record at? For you non-audio geeks, to put anything I’m going to say in perspective, a CD’s resolution has a sampling rate of 44.1 kHz at a depth of 16-bit (an 80 minute CD at 44/16 is roughly 700MB of data). In the audio world, higher sampling rates (48, 88.2, 96, 192, or 384 being the other common options) and more depth (either 24- or 32-bit) is generally a better thing. Not surprisingly, that all chews up a lot of space the more resolution and the higher the bit depth you go. They record at 48/24, and everything is stored on a server with a lot of space. Considering how much audio they must have for each project, that’s a lot to deal with (even at 48/24) not only in terms of needing it near term for the active movie project, but then to back it up and ultimately archive it for other uses down the road. This is not unlike a DBA needing to manage a total of giga-, tera-, or petabytes of information both in databases as well as the various copies (including backups). This is just the audio data – can you imagine all of the animation, too? Wow. Oh, and every bit of that audio needs to be meticulously cataloged, too. Much like you need to keep inventory of all of your servers, instances, and databases.

During the video restoration presentation, we (royal we here) got to talking shop a bit (probably to the dismay of the non-audio and video folks who were in my group) and I asked why they only scan in at 4K and not 8K, as well as discussed audio sample rates. The reason was similar: it’s a data problem and for some old film types, you’re theoretically not going to really get better with higher scans (MuppetVision 3D in Disneyland was scanned in at 6K if you care about such things, but that decision had a lot to do with the original format; this forum discusses this topic more if you want to see some opinions and other links). This puts a whole new spin on Big Data, eh? It may not be schemas, but it sure is a ton of information that is unwieldy to deal with.

What was also interesting about that presentation was the talk about the film stock and not only how flammable it could be (depending on its type), but no matter what they do, at some point, the originals will be completely unusable (much like how I related backups to audio which I blogged about awhile back). Digital, film, analog tape – stored properly it will have a long shelf life, but it may not … it all depends since there are so many factors that contribute to that. If you have important data that needs to live for a long time, preserve it properly and have adequate redundancy, too. Archiving is no joke. A company like Disney is built on its history, and they need to preserve it.

Shifting gears a bit, the Animation Research Library as I said houses 65,000,000 pieces of art. I didn’t ask, but I could tell it was all cataloged and scanned in (I thought I heard someone say 150 dpi which was good enough years ago; I believe they’re working on scanning in at either 600 or 1200dpi now that tech has moved on and storage has gotten cheaper). Anyway, point is, we’ve got another data thing going on here. This is data not only used by animators today that needs to be accessible (hello HA!), but quickly and organized. Side note: cool perk of working there is you can apparently choose pieces from the archive to put up in your office. Drool. This is Art on Demand on steroids! Where do I sign up at HR for framed pencil drawings of “The Band Concert“?

Would I still love to be an animator? Maybe, but some of what goes on at Disney isn’t too far off from what I do today. I help customers ensure their data is available when they need it, be it near line or offline. Your data is your IP … and legacy. So maybe I really did choose the right career path after all – just getting a glimpse of how Disney deals with things like archiving and redundancy gives me inspiration in what I do right now, and is more than just a doodle on a page or a ride in an amusement park.

July 31, 2012

So What’s on the Dance Card?

It looks like the next few months are going to be busier than ever. I feel like I should be asking “Where in the world is Carmen Sandiego?”, but substituting either my name or Ben’s in. Besides our normal docket of customer work, below is a list of where you can find us over the next few months. We may add a few things to the plate, but the list is everything we know of right now. Keep your eye towards our Upcoming Events page for the latest info!

SQL Saturday 158

Both Ben and I will be speaking this weekend at SQL Saturday 158 in New York City. SQLHA is sponsoring the event as well. Ben’s talk (“Surviving Your Peak Database Load”) is at 1:45PM, and my session (“SQL Server Dos and Don’ts: Keys to DBA Nirvana”) is at 4:15. I’m super excited to be doing a non-HA talk which is rare for me.

SQL Server Foundations 2012

Ben and I will be doing a two day eLearning Seminar for Penton and SQL Server Pro on SQL Server basics on September 11 and 13. It will be six sessions over the two days (three per day). This is similar to the event I did for them last year and is in the process of being updated; we’re just not rehashing the same old stuff. Full information on what we’ll be talking about (including pricing and registration) can be found here.

24 Hours of PASS

Ben and I will be doing a session at 24HOP. PASS hasn’t announced any dates or times, but we’ll be there!

PASS SQLRally Nordic 2012

I’ll be heading back to Europe for the second time this year to speak at PASS SQLRally 2012 Nordic. I’ll be doing a pre-conference seminar (“AlwaysOn: Maximizing High Availability with SQL Server 2012”) and most likely a session at some point during the other two days.

PASS Summit

Ben and I will also be at Summit in November. He and I will be doing a pre-conference session on Tuesday, November 6th entitled “Real-World System Design for the DBA: Private Cloud and SQL as a Service”. Ben will be blogging soon with a longer description of what we’re going to cover since we’ve had a few questions (mainly due to the fact we were limited in the number of characters we could use). I’ll also be co-presenting “The Good, Bad, and Ugly of Deploying AlwaysOn Availability Groups” with Mike Steineke and David Smith during the Summit (time and day not scheduled yet).

Training in Australia

In conjunction with Wardy IT Solutions, I’m going Down Under to deliver two weeks of training after PASS Summit. The class is a 4-day mission critical one complete with labs in both Brisbane (November 19 – 22) and Canberra (November 26 – 29). There’s an Early Bird discount until August 30, 2012. Click here to register for Brisbane and here for Canberra. The full course outline and details can be found here in PDF format. I’m very excited about this opportunity – both weeks should be great. I’m starting to talk with some other folks about other deliveries in 2013 of the course with labs outside of Australia, so stay tuned. If you would like to see it in your area or want to host it, contact us.

July 12, 2012

New Offering: Reviewing and Editing Services

Are you an author of a book (be it a chapter or looking to do the whole thing), whitepaper, or other type of publishable document? Are you writing a presentation and looking for a better way to put it together? Having trouble with writing session abstracts for possible speaking engagements – or ones you are being asked to do? Are you looking for an honest review and assessment to make whatever it is you are working on better? If so, SQLHA is announcing a new service for you whether you are a new writer just writing one chapter of a multi-author book or someone who has been around the block more than once.

Since the mid-90s, Allan has written countless numbers of documents from single page marketing slicks to near-1000 page books. He’s reviewed and tech edited most of those things, as well. While every publisher is slightly different in their formatting or templates, you still need to write compelling content for your readers. That is easier said than done. While your publisher (or maybe you are self-publishing) may have an official reviewer, you want someone to give you the straight scoop on your writing to make it sing rather than croak. This is where SQLHA’s new services for reviewing and editing come in.

Whether it’s your first draft no one else has seen and you want to whip it into shape before anyone else sees what it is you’re working on, or it’s later on in the process and you want a more formal review, we can help. Some of the things that we look at include:

Content – Is the content itself complelling and worthy of publishing or do you need to still shape it and mold it into something with more focus?
Phrasing – Whether you use one word too much or have odd phrasing, if your document is hard to read, no one will finish it.
Translatability – Different cultures have different references, and sometimes analogies that work well in one place are not easily understandable elsewhere. You want your document – within reason of course – to appeal to the broadest audience possible.
Grammar – While not looking to change your style, there are many basic mistakes (such as capitalization of the wrong thing, possessives, etc.) that many get wrong and go unchecked until a formal edit when these things become much more magnified. Go into your formal edits with things in much better shape.
Flow – Are things logically laid out or is it a jumbled mess? If you confuse Allan, your readers will be as well.
Goals – Are you delivering what you are intending or being asked to do? Does your document tell the whole story or did you leave out key plot holes?

The point here is to deal with problems earlier. When you’re writing a paper or chapter (or even a PPT for a presentation) that will be reviewed or seen by multiple people, much like doing sanity checks on your servers before you install SQL Server, you want to make sure things are correct and on the right path because having to do large scale revisions is much more costly as time goes on. You don’t want to have to fix flow and content issues when you’re already supposed to be published. The reviews can be simple or a more formal tech review. The document doesn’t even need to be about SQL Server, Windows, clustering, etc. – we can help you on any topic.

In addition to reviewing, SQLHA can provide editing services for your document to take it that extra step further. It allows us to be much more interactive and hands on with you to shape your end result into something everyone will like.

If you’re not an individual author but a corporation or publisher looking to have your document (of any kind) reviewed or edited, we can help you out as well.

Put Allan’s years of experience to work for you. Contact us for pricing and availability.

July 11, 2012

Anti-Virus + Windows Failover Clustering + SQL FCIs = Bad

Recently, I helped a customer plan and configure a three node multiple instance failover cluster. The Windows failover cluster (WSFC) had four instances of SQL Server (3 x 2008 R2 with SP1 and 1 x 2012 with CU2). However, when they went to go enable their anti-virus on the cluster nodes (Symantec if you must know) and set the exclusions I document in my book, things went south. Pretty much everything went offline. They were able to get pretty much everything up but SQL Server Agent for one instance just wasn’t working. I just got off a troubleshooting call with them and I think we fixed that (I’ll know for sure later once they do a few other things). If you think this is an uncommon event, it’s not as evidenced by this quick Twitter exchange I had:

With most customers, anti-virus is a standard piece of kit to install on all servers no matter what. The base rationale makes sense. I mean, who wouldn’t want to protect your servers from potentially bad stuff, right? Here’s where I lose the plot a little bit. First and foremost, your production servers are not desktops. Joe and Jane Administrator should not be surfing the ‘net from there. Assuming that is the case, the risk of getting things like malware is probably really small. If they’re doing that, you’ve got to fix your processes and lock things down. I’m not saying it can’t happen, but it shouldn’t. If you’re on the server and need to use the Internet to go read a support KB, that’s perfectly acceptable usage. But to check their Hotmail/gmail/match.com/watch porn/personal stuff? Hell no. Most admin work should be done from workstations with remote tools – you shouldn’t have to log onto or RDP into the server for most tasks. That drives the risk up of something going wrong.

The other aspect of this is really file-based. Think about it – we get documents and download files from various places. That’s arguably how a lot of the whole malware thing started. A production SQL Server server (that always looks weird to me) should not be a file repository whether it is clustered or not. I will always say that in most cases, a production boxes with SQL Server on it should be dedicated for that purpose only. I know people want to maximize usage and you can create a clustered file share, but I’m assuming here that in most cases where you deploy FCIs, it’s on dedicated WSFCs just for SQL Server. If that is the case, why would you need anti-virus especially if you’re monitoring to ensure people are not doing rogue thins on the cluster. Again, it’s easy to minimize your risk exposure with having good processes. Imagine that.

If your company is still not convinced by these rational arguments, have a quick gander at these KB articles from Microsoft:

Pay particular attention to this bit in 250355:

You can run antivirus software on a SQL Server cluster. However, you must make sure that the antivirus software is cluster-aware. Contact your antivirus software vendor about cluster-aware versions and interoperability.

Not all anti-virus programs are created equal. So make sure if you need to install anti-virus, you get the right thing. If what you have is not cluster-aware, caveat emptor.

Let me add to this cautionary tale: back in the SQL Server 2000/2005 days on Windows Server 2003, I was working with another company. We got things up and running on the cluster with no issue. I told the team responsible for anti-virus how to configure it for the clustered instances of SQL; not much more I could do. Right after we went into production, a day or two in, my databases were marked as Suspect. I did a bit of digging and the smoking gun wasn’t hard to find: right in the Event Log, it was clear as day – a virus scan kicked off at about 11AM and minutes later my databases decided to take a dirt nap from which I had to restore them from backup. They were permanently corrupted due to the scan kicking off. The admins swore up and down it wasn’t them (even though I had the proof). Things were fine for one day and I bet you can guess what happened the next day at the same time. I was livid and took this up the chain. The problem eventually got fixed, but it should have never happened to begin with. That was largely a political issue at the end of the day, but it also speaks to the fact not everyone knows about clusters and SQL Server.

This is a major reason you set filters for things like the the .mdf, .ndf, and .ldf files even on standalone SQL Servers. This is magnified on clusters where if you have an instance that fails over to another node that now takes ownership of the disks associated with that instance, the anti-virus program is going to now see them and say, “Awesome! New files to scan.” That will not only hold up your instance starting, but see my previous paragraph – you may get more than you bargained for. Not only may you blow SLAs, you have the potential of hosing databases. Just what every DBA needs.

UPDATE: Also check out Allen Kinsel’s (Twitter) blog post SQL Server 2008 & IPV6 vs Symantec. It shows where Symantec does something even more fun: endpoint protection. As my friend Nic Cain (blog | Twitter) pointed out to me, endpoint protection can be even worse than anti-virus on a clustered configuration – especially since IPv6 is enabled by default on Windows Server 2008 (and up) and you shouldn’t disable it. Allen’s post is probably why my customer’s cluster went south to begin with, and SQL Server Agent for the one instance was just a casualty.

My recommendation to you: DON’T PUT ANTI-VIRUS ON SERVERS WITH SQL SERVER INSTALLED ON THEM AND ESPECIALLY IF THEY ARE CLUSTERS. You’ll be much, much happier.

« First ‹ Prev 31 32 333435 36 37 Next ›Last »

Categories