Virb Blog

Over the last few weeks we have experienced several periods of unplanned service interruptions on Virb. This, of course, affects all of us here, but it’s even more unacceptable for our customers. The purpose of this post is to explain what happened during our last downtime (Wednesday, November 30) and how we’re preventing it from occurring in the future.

At around 7:02PM EST on Wednesday, November 30 our monitoring system notified us that Virb was unaccessible. All of us were online and feverishly looking into the problem within a matter of minutes.

After some investigation the root cause appeared to be that our original settings for PHP on our web nodes were no longer suitable for the amount of growth in traffic we’ve received in the last year. We’ve ensured that our PHP settings are now set appropriately. Also, more monitoring is being put in place to keep watch over this type of issue so they can be seen before they become bigger problems. 

Unfortunately, one thing that exacerbated the issue was our internal Stats tracking. We turned them off over the weekend to ensure that we had solved the problem. This morning we turned them back on.

This outage was not acceptable. We never want to put our customers through something like this again and we’re working as hard as we can to make sure we won’t ever have to. The level of patience we received from our customers—those who had every right to be frustrated with us—was amazing. We’re glad that you’ve chosen Virb to build your home on the internet, and thank you for your understanding.

As always, you can follow @Virb if you’d like to receive status updates via Twitter.