Our website was down for nearly 6 hours on Tuesday. We received no advance warning and there was very little communication from our web host during the outage. As you can imagine, we were a bit flummoxed trying to determine the cause and navigate the consequent lack of communication. This morning we received the following email from the COO of Volusion, our webhost:
As you likely noticed, we experienced a critical hardware failure yesterday that resulted in your storefront not functioning properly. We take full responsibility for this incident; our mission is to help you succeed online, and yesterday we didn't live up to that promise. We apologize for the issues this caused you and your customers.While this update provides much more insight into the problem that severed our connection with our online customers, I can't help but feel that a little transparency from the beginning would have saved a lot of anger. Not to mention the subsequent finger-pointing that occurred on Twitter as those who were affected desperately tried to get immediate answers using the only medium readily available to them (A very public medium).
In addition to the technical issues, we also did not communicate with you throughout the incident as we should have. In our efforts to get your store back up and running, we neglected to keep you informed.
While I cannot promise that problems will never arise, I can promise that we are committed to mitigating future issues in a way that minimizes the impact on your business. I've attached a technical explanation of what occurred yesterday to the bottom of this message.
Sincerely,
Clay Olivier
Chief Operating Officer
Volusion
Technical Root Cause Analysis
Early yesterday, one of our blade server chassis lost connectivity to our Storage Area Network. This resulted in many of our customers experiencing downtime or errors on their storefronts.
Upon identification of the issue, we immediately brought HP on-site. After extensive troubleshooting, HP engineers diagnosed the issue as a failed mid-plane within the chassis. The engineers replaced the mid-plane, and all connections were restored.
This is the second HP mid-plane that has failed in the past 30 days. While the chances of two failures of a similar nature are very remote, HP was able to confirm this to be the case. Based on the serial numbers of the two failed components, we have reason to believe that both came from a bad batch. We are researching this further with the help of HP's labs, as well as performing a full audit on all of our HP equipment
We recognize the impact that yesterday's outage may have had on your business. We are working diligently with HP to prevent a recurrence of this issue in the future.
What's your company policy? Do you believe in being transparent up front or do you prefer to fly by the seat of your pants and follow up with damage control later?
Allison B. Kontur
www.BathBodySupply.com
Posts
2 comments:
As a small business owner, it is crucial that I be honest and forthright with my customers-especially when things go wrong. We're human and make mistakes, but pretending like nothing ever happened or trying to do damage control after-the-fact does not make you look good as a business. It's a sure fire way to lose customer confidence.
Patrice: Thanks for sharing your insight!
When our businesses are new and we have a problem, it's easy to want to avoid the issue (especially if you've ever worked for someone else and you are used to a corporation shouldering the burden for customer satisfaction).
However, experience teaches us that if you nip a problem in the bud when it first arises, it saves a lot of hassle later on. Sure, your customers might still get mad. You might even lose their confidence. However, if you make it a priority to be forthright and explain how you intend to prevent issues in the future (with real solutions), your customers will not only thank you, they will trust you. And in business, trust is everything.
Post a Comment