Monday, November 24, 2008
Backup Plans. A Must! Assembla and GoGrid At Fault
We were thrust into the unwelcome position of creating plans for alternates for two of our services. Neither was at our urging, but it's probably a good thing we encountered these situations early on.
The first was our svn respository. For those unfamiliar with coding, it's really just a place to store all of our code so we can reach it at all times. It also lets several team members work on the same code at the same time, managing all of the different versions. It should be mandatory for any coding project. Even those with just one person. Because this repository stores history too. So if you make a change, then a few months later determine the change was a mistake...you can just unwind your changes.
We were using an online service called Assembla. At first, I have to admit, I was surprised it was free because it was such a kickass service. That didn't last long though. A message was posted on their blog about changing their free service to a subscription model. If we wanted it to remain free, we'd have to allow our code to be visible to the public. Naturally, this wasn't an option. Unfortunately, we don't read their blog. Why should we read the blog of an svn service? I don't think anyone else read it either, because a few weeks later, when they realized this, they sent emails to people using their service, advising them of the changes. Great timing as we were about to push out our alpha. We need our code. In SVN, readily available.
We weren't the only people pissed about this. I'd be fine going into their service if I knew I had to pay. But they advertised a free service, got our business, then decided to flip a switch. Comments on their blog skyrocketed. From both sides of the argument. One side said "hey, it's a good service, a charge is fine". The other, "we felt deceived, we don't have a choice now that you have our code, and there's no way to be grandfathered in since we helped build the product and community".
The flaming continued. And in several instances, their CEO, Andy Singleton, vociferously called out several members, reciting personal details, and hurling insults. I really couldn't believe it. Definitely not the way to conduct yourself. And not the way to make me feel comfortable about having my code on their site. What if he got pissed at me? Would he just steal my code or "accidentally" make it public?
I wrote. He responded. And told me how trite it was for me to complain. How I should just stop serving pizza once a month for my team and pay for the service. Yea, that's the way to my heart. Let's see. I believe it's $2/user/space/month. And $3/Gb/month for space. Since we have 4 team members and 3 spaces at the time. Hmmm...if my math is correct, that turns out to $24/month just for our users. Then the space. Probably be another $2 maybe. So $26/month. And what happens when we have a few more eng's to add on? Like we do this week. We are a bootstrapped startup. Any costs we can save, we will. And $32/month is not worth it. We have a small project. We don't use it. They'd be making probably $30/month from us. So we moved our repository to our own server and are using Redmine, which I'm incredibly happy with.
The next backup service issue has to do with our servers. Yesterday our servers on GoGrid were mysteriously rebooted. According to GoGrid, there was a node outage. And since our virtual server sits atop this node, we were affected too. Then the root user entry in our user table in MySQL was removed. I have absolutely NO idea how these could be linked. But it is strange. I just got off the phone with GoGrid. They're definitely bright and proactive, but this outage isn't consistent with their 10,000% uptime guarantee SLA. They'll be sending out an RFO (reason for outage) in the next day or so. The question remains, how reliable will they be? They're a startup too - can we trust them? I always like to give the little guys a chance. As long as it doesn't impact business, I think we'll be ok with them. For now.
UPDATE: GoGrid has been great. They just called me back to talk to me more about the issue. To ensure we're covered under their SLA, etc. Good customer support.
Lesson: Have a backup plan. Or, at least think about one ahead of time. And don't screw your customers.