Blog

November 21, 2010

A Conversation re: Downtime

Filed under: All Blog Postings — Craig Pollack @ 6:27 am

I recently had a rather interesting conversation with a client about downtime.  Actually, it didn't start out as a conversation, but rather as a directive.  Our client was communicating (quite emphatically no less) that they couldn't be down at all, ever.  Not for maintenance.  Not for upgrades.  Let alone as the result of a disaster.  Never.  Literally 5 nines (ie: 99.999%) of uptime wasn't good enough.  I understood what our client was asking for, unfortunately I don't believe they understood it.  Keep in mind (and other than his perception of his own accessibility), we're not talking a mission critical system in health care or life support systems or a business that was taking orders 24x7.  It was a "regular" brick and mortar business with pretty much your standard hours of operation.  Essentially, it was just that the owner never wanted to be without access to his system whenever he wanted it.

Again, I certainly understand this need (many of our clients never want to be without access).  However, while this in and of itself isn't unreasonable, what was most frustrating was the lack of any understanding or openness to work as a team to be able to achieve this for them.  The problem wasn't that they couldn't have any downtime.  The problem was that they didn't have a clear understanding of what this really meant and what this really takes.  They're simply under the perception that "our system has never gone done before, therefore it should never go down in the future".  And the vibe I was getting was if it ever does "go down", it's our fault.  Keep in mind, this is a network we inherited and had only been supporting it for a short period of time.

I've found through the years that one of the most difficult situations to overcome is building a trusted relationship while the perception of the business owner is incorrect as to the state of their network - especially during that growth phase from a small business with 1 or 2 servers and like 10 workstations to a multi-server network with around 25 - 35 users.  While the business systems and related processes have become more complex and the underlying network infrastructure has also grown more complex, the perception of technology within the business hasn't kept pace.  Many business owners seem to have trouble with this change.  More maintenance is required.  More sophisticated tools and management processes are required.  And more expertise and understanding as to the role technology plays in the overall business plan is required.  Yet the business owner views things statically.

It's not unreasonable to want uninterrupted service.  What is unreasonable is to think that it doesn't come at a cost - both in time, understanding, and the added complexity of redundancy.  This is one of the more compelling arguments for moving things "to the cloud".  However, the key point here is about the perception of technology, its related complexity, and the underlying costs. 

Getting back to the initial conversation, the problem wasn't that they couldn't be down.  The problem was that there were so many points of failure that our client didn't see and acknowledge and therefore wasn’t willing to address financially.  Little or no proactive maintenance.  No down time for server patching.  No added redundancy utilizing virtualization.  Somehow the word "backup" got translated into meaning they're protected against any downtime.  Again, it was his antiquated mindset that was getting in the way - they never went down before, so how could they be down in the future? 

It's interesting how much businesses have come to rely on technology, yet the mindset change needed to effectively run it hasn't kept pace.  The way we see things, as one of our clients' key trusted advisors it's our responsibility to help them bridge this gap. 

So, has your mindset about your technology kept pace with the change of your technology?