On Your Six, Boss

Quick note that post-by-email and comment-by-email is (sometimes?) failing silently without actually posting right now! I'm pretty sure this is related to last night's shenanigans and will be fixed once Mark can finish the full fix for it, which he's working on, but if you've posted or replied by email in the last 24 hours, fish it out of your sent folder to check if it posted!

EDIT: This should be fixed as of around 7AM EDT! We *believe* everything that was stuck in the plumbing has been sent along to your journal or the comment thread it was meant for; it's definitely not where it was stuck anymore, at least.

Robby has managed to put in a temporary fix for the site errors and things failing to refresh or not showing up where they should! The permanent fix is going to need Mark's experience, and unfortunately -- seriously, this literally never fails -- Mark has been on an international flight all day, because of course he has. (Never. Fails. He and I are not allowed to both take vacation at once.)

The site will work just fine with the temporary fix in place, things just might be a little slow here and there. We'll keep you updated.

It's been a while since we've done a full code push rather than just hotfixes for bugs, so we are well overdue! Depending on availability, we're aiming to do one sometime soon; we'll let you know specifics once we've worked out good timing for everyone who needs to be available.

However! The reason it's been so long is we kept trying to get some of the stuff that's pending to "really finished" instead of just "mostly finished", and then we once again looked around and went "oh no, this is a really big code push with a lot of changes". Those make us nervous, because while we do a lot of testing ourselves, y'all are really creative in how you use the site and we inevitably find a bunch of edge cases when we let you loose on new code with your real-world data!

So, if folks have some spare time in the next few days, it would be a huge help if you could spend half an hour or so using the site the same way you normally do but with the "Site-Wide Canary" beta features flag turned on. Canary mode is a sort of "live testing" mode: it's your real data, but running the most up-to-date code.

Canary mode always does have a few glitches -- there may be missing text strings or errors about missing database properties, which is a limitation of how we run it. We don't need to know about those, but anything else weird that you run into, leave a comment with what you were trying to do and the error message you got.

I'll repeat that the "here be dragons" caution that's on the beta features page: some things may be broken, so don't use it for when you're doing something important. But a few more eyeballs on it before the push will help the push go more smoothly for everyone.

For folks who want to concentrate on what's changing, we haven't finished the second code tour of what's going to be in this push, but the ffirst one has a good chunk of what's going to be going live. (We'll get the second half done ASAP!)