Thursday, January 29, 2009

VPSLAND killed Foldees, scared the crap out of us, and brought us back

At this point, we're struggling to find the appropriate words for the usually good-natured Foldees blog while dealing with the current situation. As you can see, we've already failed with the blog title.

The past 48 hours have been the scariest of our brief existence as the world's only designer-created greeting card site. Here's what happened:-

-------------------------------------------------------------------------------------------------------------

About a month ago, we had some serious outages with our site due to a highly unstable server provided by our hosting provider, VPSLAND. This resulted in our site being down during our crucial contest submission, which we then had to extend.

During this period, we were in frequent contact with VPSLAND to get to the route of the problem, and they suggested migrating over to one of their more "stable servers". We came to an agreement that we would get 5 days to perform the migration, while VPSLAND kept both the old and new servers active. We decided that the low period of late January would be a good time to do this.

On Tuesday night (Jan 27th), we emailed VPSLAND to start up the new server to begin the migration. On Wednesday morning, we woke to find Foldees.com was unreachable. We sent several emails to VPSLAND that day (they have no phone support, which should be a fair warning) to restore the site, but they only sent one support email back saying that the site was fine and that the connection was "verified".

However, as you guys know, the site has been down for the past 48 hours. After three days of frantically emailing our hosting provider, the Foldees team went through some of its darkest hours in our brief existence. We feared the worst, that the server instance had been deleted and that we would lose our database.

Would we be able to recover the database? What about the contest ratings and comments? In this time, they did not reply a single one of our panicked emails as we set up a quick program to continually ping the server to let us know if it ever came back to life.

Half an hour ago, the site magically came back up. I'm not by any regard a religious person, but at moments like this, one has to thank the big guy upstairs. Also, even though we'll be leaving you very soon, thanks to VPSLAND for at least bringing the site back up (even if it was your bloody fault in the first place). Quite a few of you have sent your well wishes to us about the site, and we'd just like to thank you as well. Your support during this brief, but trying time has helped greatly, and we love you for it!

----------------------------------------------------------------------------------------------------------------

Anyways, just to let you know, we have already signed up for a significantly pricier (and more reputable) host called Blacknight in Ireland. We are also putting significantly more frequent backups in place to ensure that this does not happen again. While I'm on this point, let me add something of value a midst the death and despair.

<valuable advice>

How we stumbled upon our current hosting company was through searches for "best VPS provider" on Google. The lesson we've learnt here is that in future (and this is awesome advice for all professionals sourcing for suppliers), is to do the opposite. Having searched "VPSLand sucks", we found a slew of sites condemning our current provider to Internet hell. Other words we found linked to them include "incompetent", "buffoons" and "nightmare". All sentiments we can share at this point in time.

So, the painful lesson we've learnt here is that for any future sourcing for providers of anything (plugins, hosting, blog engines... etc), do make sure to search for "<provider's name> sucks" on Google

</valuable advice>

We are currently checking our database to ensure that no data has been lost in this time. If you feel something has gone missing, do let us know at support@foldees.com and we'll do our darndest to get everything back working the way it should.

Once again, we offer our apologies for your inconvenience, our gratitude for your support, and our humble hope that you will stay with our little site for a long time to come :)

7 comments:

angel said...

It's nice to be back...welcome back!

Mike Tee said...

Just an episode of bad luck I guess. Worst things have happened! In any case good to see you guys back online.

Milkberry said...

Phew. Don't forget to breathe! You guys did a great job of not buckling under pressure. Salut!

chak said...

Angel & Mike: Thanks guys!
Milkberry: We were already typing out a very very long apology letter to you guys, when our programmer called frantically to tell us that the server was back up!

But yes, we are here to stay.

And we can take our programmer off life support now. Thanks milkberry!

bem69 said...

Glad foldees is back up. And you got some great advice there, I'll keep that noted. Did you try that with the host you're migrating to?

chak said...

bem: YOU BET! Nothing bad came up... so either they're really good, or they've got the best PR people in the world. Fingers crossed. Migration will take place between Monday and thursday next week, so expect a few outages yea?

dilan said...

If you want a real VPS services you can go on :

http://digitalswift.net/
https://surgesecure.net/digitalswift/lvr.html