Hi everyone,

My deep apologies for this morning’s downtime.

It wasn’t actually a problem with the app. Instead our server’s security certificate – the one that allows you to log into ActiveInbox securely – had expired and the new certificate had not come online.

This meant that some ActiveInbox features were disabled (principally the To Dos and Notes).

Renewing certificates is an annual process, involving multiple vendors, and to be totally transparent, is one area that we don’t have a deep expertise in. I accept total accountability for the failure.

As a remedy, next year we’ll engage a consultant with the required expertise to handle this for us, to prevent it happening again.

Clarifications

Delayed Communication

We didn’t first tweet it until 11am GMT. This is all rather mundane detail, but to be as transparent as possible… This morning I elected to work from home for a few hours, where there’s currently no internet, to allow me to deep-dive into some product work without distraction. Normally this is fine, but obviously today everything went wrong. Lisa, who’s not technical, finally realised the breadth of the situation at 10:30 and rose the alarm with me, at which point I came into the office. All problems were resolved by 12:30 GMT.

I’m totally empathetic that the lack of communication was particularly difficult, and as a remedy:
1) I’ll make sure my personal mobile is pinged if the server is unreachable.
2) Lisa will be emboldened to raise the flag sooner in future.

ActiveInbox’s Feature Dependency on the Server

ActiveInbox is in fact already modular, so that when the server communication is lost, the only things that *should* be disabled are To Dos/Notes/Ranking (i.e. the bits that absolutely need the server).

I gather that on Firefox especially, for some people ActiveInbox’s functionality loss was more severe. I’ll be extensively checking the failure mode.

I’m also going to increase our ability to communicate in-app during downtime, especially for notification that everything is ok again.

Categorised in:

This post was written by andymitchell9496