Well, it's been a bit of time since the last time I posted, and well, I had to think a fair bit on the comments I received. It's become very clear that while I'm still willing to at least help in technical matters, the effort to reforge SN is much higher than I expected. In addition, given the, shall we say, lukewarm response I got to my posts and journal entries, well, I'm clearly not the right person for the job.
I think at this point, it's time to figure out who is going to lead SN going forward. After my de facto stepping down in 2020, the site has, for want of a better word, been a bit listless. At the moment, no one on staff really has the cycles to take that position on. A few people have expressed interest in the position, and I've talked with Matt, who is co-owner of the site about this. By and large, whoever fills the seat will have to figure out what, if anything, needs to change in regards to moderation policy, content, and more.
If you're interested in potentially fulfilling the role, drop me an email at michael -at- casadevall.pro, with the subject of "SN Project Leader", and include the following:
I'll leave this call for candidates open until December 14th, at which point Matt and I will go through, and figure out our short list, I'll talk to editors, and solicit more comments from the community. I'm hoping to announce a successor in early January, and formalize the transition sometime in February, which will be the site's 9th anniversary.
(Score: 1) by shrewdsheep on Monday December 05, @07:04PM (1 child)
This + using either a relay log or the general query log which should be rotated at the time of the db dump should allow the reconstruction also to points between dumps. I have not worked with mysql and do not know whether the dump is atomic which might be required to make replaying the log-files work.
(Score: 3, Interesting) by RS3 on Monday December 05, @07:21PM
Yes, really good point. AFAIK, and web search results say MySQL is atomic, so you'd want to back up the 2 (3?) log files.
It really depends on the time-granularity (resolution) you need. Here, I dunno. It's not a Wall Street high-frequency trading database, so sub-millisecond is probably not necessary.
It'd be good to do some analysis of various buffer / cache flushing. Big buffers can result in bigger data loss, but too small buffers would bottleneck a busy system (make faster storage system) so some tuning is in order to figure out the compromise, but that's pretty easy.
Incremental backup is available within MySQL, so I'd look strongly at fairly frequent incremental backups, which again, compressed text is going to be tiny, and you could auto-delete older ones after a full backup is done.