Hi everyone,
You’ve probably already noticed if you’re reading this, but an outage is preventing ActiveInbox from loading (starting overnight 5th/6th November).
We’re going as fast as we can – and my extra apologies for those who emailed in, as Lisa has an appointment this morning that means she’s late in and won’t have replied yet – the timing is truly unbelievable as she is famous for never taking time off! But please know that we’re 100% focused on the engineering side.
What’s causing it?
I believe it’s now fixed (see our latest update) – I’m just working to make sure the systems auto-recover from this in the future, after which I’ll declare it 100% resolved.
Original 10am GMT
It looks like a continuation/echo of the problem we had a month ago (where our server hosts changed something without notifying us first). But I’ll report back any findings as they happen.
Update 11:53am GMT
It was a continuation of the last problem (I was immediately suspicious, as it’s exactly 1 month since the previous one). Our server host changed the authorisation process to an underlying service last night, and this time it affected one of our support servers.
The good news is that the work we did last month, to the main server, has been 100% rock solid. And I’ve manually got the support server working to make sure ActiveInbox will load.
We have a long-term plan to move to a new server host, as their behaviour has been untenable (they’re the formerly rock solid company called Heroku, now owned by Salesforce no less – but the acquisition seems to have become a low priority for them, and lots of their other customers are vocally moving away, as will we).
But to make that process smooth, it’s going to take some time.
So until then, if this issue occurs again I want ActiveInbox to auto-fix itself. I’m working on that now.
Update at 12:20pm GMT
I spoke too soon – the major problem is solved, but there’s something else that’s still affecting people. I’m digging in again.
Update at 12:33pm GMT
It appears part of our server host has become inaccessible (starting suspiciously at 11:50 when I think we solved the initial problem – and its the same underlying service that’s affected). I believe this to be a problem at their end, but rather than understand it I’m just going to see if I can replace that component to get you back up ASAP.
Update at 12:41pm GMT
All is OK again! Judging from our metrics, errors have fallen to 0… so I think I’m safe to say we’re back. (Let me know if you still have problems).
Final Update
I’ve successfully set up auto-recovery, for this specific issue at least, so it won’t happen again.
What’s the workaround?
ActiveInbox’s organization system is based entirely on Gmail labels, so you can just use labels to move around.
For example, today is ZD/20241106 (you can find this label in Gmail’s left hand sidebar).
(For reference: ZD/ is a prefix to tell ActiveInbox it’s a date; then it’s formatted as yyyymmdd… so 20241106 is the 6th November 2024.)
You can also add Gmail labels manually, e.g. to set a deadline for tomorrow, add a label for ZD/20241107.
(My many many thanks to Alex commenting with this tip – in my rush to get things fixed I’d forgotten to add it)
ETA on the fix [Now fixed!]
Original 10am GMT
I’m currently investigating, and while I don’t want to tempt fate, I think this will be quick to resolve – possibly even within the next hour (11am GMT). Maybe.
Update at 11:53am
ActiveInbox is actually running again – if you restart Gmail.
The 11am eta had been a little optimistic – it took me until 11:53! But ActiveInbox will now load. I’m not yet ready to declare it fully fixed, as I need to make it auto-recover for this type of problem before I’m fully satisfied. I’m doing that next.
Update at 12:20pm
There’s still a possible issue occurring – will update ASAP when I’ve understood it.
Update at 12:41pm
I think the secondary issue is now also solved. I’m back to working on auto-recovery. Try refreshing Gmail and ActiveInbox should load.
Final update
Auto-recovery is now in place for this specific issue, should it recur.
Andy
This was written by Andy Mitchell