Stories
Slash Boxes
Comments

SoylentNews is people

Meta
posted by NCommander on Monday May 22 2023, @04:00PM   Printer-friendly
from the end dept.

This is the post I never thought I would have to make. I am also writing this post on behalf of SoylentNews PBC, the legal owner of SoylentNews, and not as a member of the staff or the community.

SoylentNews is going to shut down operations on June 30th.

This wasn't an easy decision to come to, and it's ultimately the culmination of a lot of factors, some which were in my control, and some that weren't. A large part boils down to critical maintenance to the site not properly being performed for a very long time. To pay back the mountain of technical debt we've built up, it would require relaunching the site from scratch.

I'll discuss this more in depth below, but I can't personally justify the time any more, especially due to the negative impact that SN is having on my personal life.

Before we shut down, at least for the foreseeable future, I'm going to outline the situation as I see it, my own personal responsibility, and what happens next.

Technical Reasons

Let's start with the technical nitty gritty. SoylentNews was, in November 2022, at the point where it was about to have a fatal database crash. The database cluster was wedged in an invalid state. Backups weren't properly being done. As it was, we lost several days of postings after a hard crash. On top of that, we had multiple public facing machines running outdated versions of CentOS and Ubuntu running net facing services.

There's two distinct problems here, both of which have to be addressed.

The first is the site itself, and specifically what we can or can't do with it. At this point, rehash, the backend that runs SoylentNews, is nearly 30 years old, and it was written in what can be generously described as angry and especially esoteric Perl. Perl was already going extinct when we launched in 2014, and at this point is mostly relegated to legacy backend code which is slowly but surely disappearing.

Complicating matters is that rehash is specifically tied into Apache 2.2 via mod_perl, a version that is well past end of life and significantly out of date. While the website is well sandboxed and battle hardened, running obsolete code on the public Internet is not a smart move especially when combined with many of the other factors listed below.

To just keep up with patched software, we would either need to port the site to Apache 2.4 and a recent version of mod_perl, or break the Apache dependency with an alternative like FastCGI. This would also require updating the base version of Perl5 to something more recent and hoping all the necessary CPAN modules have either been updated, or can be reasonably replaced, which is doubtful at best

As the person who actually did the base port of rehash from Apache 1.3 to Apache 2, this is a massive project regardless of which way we would go. This would require a full rebuild of the /srv/soylentnews.org directory which alone could easily take weeks or months of work. That doesn't take into account all the other bits of infrastructure and software that would need to be reworked, rebuilt, or replaced.

When we migrated to rehash, we had more staff who could QA the site and quite a bit went wrong trying to do that relatively simpler migration.

As we are now?

I don't see how it's possible anymore.

After everything that has played out, I'm having trouble working up sufficient motivation to work on the site and bring it up to a serviceable state when combined with the amount of friction I've experienced just getting us here.

I had hoped to hire outside help or at least raise enough through livestreaming other SN related work to offset the costs. At this point though I believe that is a lost cause as well. If this was the only major problem, it would be bad enough. Unfortunately, it's just the tip of a very large and very ugly iceberg.

The deeper problem is that everything else has bitrotted over time.

SN's backend is something of a jigsaw puzzle which is documented in one of three places: on the site, on the internal technical wiki (which is currently down), and on the old public facing wiki which is also down. None of that documentation was or is consistent with the actual state of reality, and quite a few parts, like the MySQL cluster, were somewhat esoteric.

In practice, if you want to know how anything was plugged into anything, it was a matter of pulling cables and figuring out what broke. It also doesn't help that the backend is notoriously noisy. That makes it hard to sort out real errors from the chaff. This was a large part of why the Zoo plugin, which does the sidebars, was broken for most of December. It also didn't help that we had three different OSes (Ubuntu, CentOS, Gentoo) which complicated system administration.

Furthermore, there have been major disagreements among the sysops on actually doing any major upgrades. Someone would complain that we should do something. There would be a lot of arguments about it. In most cases, nothing got done. Because of this ongoing friction, it became increasingly more common for no one to install updates. This is why we never upgraded from Ubuntu 14.04 to 16.04 back in 2016. I eventually said we should just go to Gentoo, since there was a widespread belief that upgrading the distribution would break everything. This suggestion ultimately just ended up with only our development machine on Gentoo, and that too was woefully out of date.

When I finally checked in in November 2022, after two years, the site had finally reached a breaking point. I talked with some people in #chillax, and I got a state of affairs from mechanicjay, and I decided to do what should have been done long ago.

Clean house.

I didn't ask for permission. I didn't wait for people to answer DMs. I just did it because we had done this go around one too many times in the past.

I will let the community decide if I was justified or not in doing so.

A lot of this involved installing over a decade of upgrades. Setting up and configuring firewalls and removing unneeded services. Backing up and decommissioning old boxes. Given the extended period of time without updates, you can imagine that I at least have some concern at the number of potentially vulnerable backend services that were exposed to the Internet.

I found no evidence of breach, but given the period of time, and general lack of maintenance, I am at best uneasy.

I could have done better.

In the end, I finally installed almost a decade of upgrades in December of 2022, but that only postponed the inevitable. I also trimmed the number of machines and services in an effort to be at least slightly more secure on the Internet. However, ultimately, without a way to bring in new users, SN is slowly going to attrition itself to death.

Some might argue that I simply let it be, or should have let it be in November, but I really did hope that I could pull it out of this death spiral. Over the last few months, it has become clear that the only way work is going to be done is if I do it or if a miracle happens.

The problem is: as part-owner, where do you go from here?

There's also the matter of liability.

Ultimately speaking, if something happened with SN, Matt and I would be jointly responsible since our names are on the legal documents. I tried to find someone to take my place, and failed. I am legally attached to something that is barely being maintained, and frankly, I can't carry this cross any further.

My Role In This Outcome

I guess this falls down to a lot of my personal responsibility. While SN is at its heart a community project, it is also a business, one for which I have served as its president for its entire life. I really had no idea what I was doing when we started, and this had long term effects on SN as a whole. Part of this was that we only had subscriptions as a revenue stream.

Without a more solid revenue stream, the PBC was essentially hostage to the small trickle of money subscriptions. In a volunteer organization, it's a matter of "who shows up to do the work" dictating the direction of the site.

In the early days this wasn't a problem, I had plenty of free time, and people were often willing to help. That's largely how the site got ported to Apache 2, and why we were able to stay up for more than a year. Meanwhile, solving UTF-8 support was one of TMB's and MartyB's projects. As the early enthusiasm died off and staff began to leave, essential tasks were becoming less and less likely to be done.

That ultimately created a negative feedback loop in which technical debt continued to pile up.

SoylentNews also doesn't have a growing community, partially because we have very few inbound links, and are fairly low in search results. In our early days, folks followed us from Slashdot, and some viral posts on places like reddit and HackerNew did help to build the community, but this has largely evaporated.

Growth of some sort is important because communities have a natural attrition rate. People leave, die, or otherwise go inactive. Year over year, the community has shrunk, primarily because we don't bring in a lot of new blood.

Furthermore, the Internet as a whole has changed. When we started, GamerGate was yet to happen. The world couldn't even imagine the rise of the Trump presidency. In theory, the moderation system should have been able to handle disinformation, but the mod system requires a certain critical mass to work. Slashdot's mod system could only work as it does on a large community, and we found at least one critical flaw with its base assumption:

People rarely if ever downvote.

This, combined with ineffective anti-spam meant that it was relatively easy to game the system, and allowing bad actors outweighed good. My perception was SN's signal to noise ratio was becoming more noise year over year, and there were many conversations on this, which ultimately went nowhere. For me, personally, it finally reached a head with COVID. The amount of medical misinformation and similar such disinformation got to the point that I felt we needed to drastically overhaul the site.

This lead to some very bitter arguments.

Ultimately, I was overruled, and I attempted to resign after bitter arguments in the staff channel. My resignation was written, but ultimately never posted, and I left on bad terms with the staff at the time. Consequently, I remained President of the PBC. At that time, I requested Matt remove me from the position, but we never formalized this, primarily because there was no one to replace me. It should also be noted that we were missing a secretary and unable to find a replacement after mrcoolbp withdrew due to personal life reasons.

I could have, and perhaps should have, forced the issue then, but I could still remember how the domain was hijacked in our early days, and didn't want this to be a case of sour grapes. I also had a reasonable belief that SN would still be maintained by the active staff. I turned my attention towards my other endeavors such as my YouTube channel and tried to put it behind me.

Two years passed.

That was not the end of the infighting, and that ultimately led up to TMB leaving in 2021. The site was now running with the bare minimum of maintenance mechanicjay supported by audioguy could give it with no hope of a long term solution in sight. Had I not checked in, and decided to do emergency maintenance, the odds are that it would have been a matter of weeks or months before a severe system crash would have irreparably corrupted the database.

As it was, we had two hard crashes that lost weeks of posts. There were no functioning backups that I could find.

I did two emergency rounds of maintenance that saw the backend database replaced with a standard vanilla MySQL instance and drastically downsized the number of machines, cutting the monthly bill more than half.

However, it's become clear to me that this was too little, too late.

Many of the issues that were present when I resigned were still here. At the end of the day, I found myself caught between my responsibility towards my site and my own frustrations for what it had become. This combined with a personal disaster in my life starting in December meant that I had very little time for SN.

This was also combined with the dawning realization of how difficult it would be to get new sysops and devs to replace myself and those that had left. While I was willing at least to put some of the legwork in, no one really wanted to sit down and help with the business side of things. It felt like everyone else decided we should all hum loudly. While we had some volunteers for sysops, my lack of time, combined with the relatively arcane nature of our backend mostly nothing being done.

I honestly don't know if there was one specific misstep that led to this outcome. However, the need for sites like Slashdot and SoylentNews was already passing when we launched. Slashdot is a shell of itself, and most of the role of news aggregator is taken up with sites like reddit and HackerNews. The need for something like SN has largely disappeared.

That means for SN to exist, it has to exist for itself, and well, that's the rub of it. SN stopped being maintained while I was absent. It wasn't being well maintained well before that point. It's not going to be maintained now simply because I can't justify the time and effort anymore and no one else is putting time or effort into this either.

Suggestions like running ads to try and pay for some of the maintenance costs have either been rejected or at least treated with enough skepticism that makes me doubtful it would help.

Finally, I'm tired of fighting over every single issue which in the end leads to nothing being done and everyone just walking away unhappy.

What Would Have Been Needed To Save SN?

As before, I'll break this into two sections, involving the technical, and the non-technical. To summarize, it essentially required people to take responsibility and pledge to fix it as well as relieve me from my position from the PBC.

Technically speaking, we'd need to be able to refresh the site infrastructure as well as the site's backend dependencies. You're essentially dealing with a legacy Linux install that has been upgraded from Ubuntu 12.04 to Ubuntu 22.04 that at least a dozen of sysops have worked on.

To reduce site admin burden, we'd probably end up migrating email, and most services beside IRC and the website to third party hosting providers. This would have solved many of the email and registration issues that have plagued the site since GMail made their spam barriers extremely hostile to external SMTP hosted mail.

We would also need a development environment that properly tracked with production to allow changes to be done incrementally and rolled back, something that was a continuous problem throughout every major site upgrade. This would let us test each aspect of the overhaul and deploy it piecemeal instead of having the site be broken for weeks or months as happened with the much smaller November upgrade.

Ideally, we would use an automation deployment solution such as GitHub Actions which would make sure the machine state was always in sync with the build files, and allow for easy and rapid deployment of backend patches and security updates.

With this all done, it would have allowed site maintenance to easily be done en masse to all machines and without risk of the site breaking in new and arcane ways.

I did talk to Matt about the possibility of either fundraising or selling stock in an effort to finance it.

I also made multiple efforts to find someone who was willing to seriously take over the site and take over the PBC. There were a few email discussions that went ultimately nowhere.

What it boils down to is that to do anything with the site, I would have to put in legwork that, after everything that has been said and done, I am no longer willing to do nor is anyone truly stepping in to try and take over for me.

It doesn't help that nearly every single thing I've laid out here was shot down by at least one other member of staff while at the same time no realistic alternatives were worked on or even proposed.

What Happens Now?

At this point, we need to get the expenses of the PBC to zero. We have about $1,500 USD in the bank, most of which will go to handle our shutdown fees. I want to give a window for people to exchange contact info and write goodbyes. Subscriptions will be disabled on the site by time this post goes live. SN doesn't have a robust infrastructure to process refunds, and TMB wrote most of the code involving that. I am discussing with Matt what our options here are, but in the worst case scenario, any leftover will be donated to the EFF.

A final backup of the VMs and site database will be taken and soylentnews.org will be redirected to a static page. Everything representing the site be archived and taken offline. I'm going to hold the domain name and backups in trust in the hope that circumstances in the future may allow for the site to return in some form.

I wish I did not need to say such a thing might happen, but all things must end.

Until we meet again, ~ NCommander

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 5, Insightful) by Mojibake Tengu on Monday May 22 2023, @04:27PM (20 children)

    by Mojibake Tengu (8598) on Monday May 22 2023, @04:27PM (#1307352) Journal

    If it works, do not fix it! is the rule.

    So, SoylentNews is going the fate of Kuro5hin...

    --
    Respect Authorities. Know your social status. Woke responsibly.
    Starting Score:    1  point
    Moderation   +3  
       Insightful=2, Underrated=1, Total=3
    Extra 'Insightful' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   5  
  • (Score: 5, Insightful) by ElizabethGreene on Monday May 22 2023, @04:40PM (17 children)

    by ElizabethGreene (6748) Subscriber Badge on Monday May 22 2023, @04:40PM (#1307359) Journal

    "Out of date" is political, not technical concept

    How confident are you in asserting this? We're not talking about an engine being outdated because it uses pushrods and solid lifters vs. roller lifters and overhead cams. We're talking about a website that has to respond to evolving security threats, evolving security standards, automated attacks by spammers, human orchestrated attacks by smurfs, and mod point manipulation. To assume that technology will continue to "just work" indefinitely is unreasonable.

    Unfortunately, technical debt accrues interest.

    • (Score: 5, Insightful) by Mojibake Tengu on Monday May 22 2023, @05:17PM (16 children)

      by Mojibake Tengu (8598) on Monday May 22 2023, @05:17PM (#1307381) Journal

      I am absolutely confident about this.

      What I observe in all kind of engineering is the new technologies are often weaker than old ones. It takes years of lifetime experience to make anything safe.

      That happens with software too every time inexperienced freshmen take over the solid robust stuff, not understanding it well, enthusiastically starting breaking things and introducing new, unthinkable vulnerabilities.

      --
      Respect Authorities. Know your social status. Woke responsibly.
      • (Score: 5, Touché) by martyb on Monday May 22 2023, @11:28PM (15 children)

        by martyb (76) Subscriber Badge on Monday May 22 2023, @11:28PM (#1307506) Journal

        That reminds me of an old adage:

        Just because we taught you everything you know
        doesn't mean we taught you everything WE know.

        --
        Wit is intellect, dancing.
        • (Score: 5, Insightful) by Reziac on Tuesday May 23 2023, @02:38AM (14 children)

          by Reziac (2489) on Tuesday May 23 2023, @02:38AM (#1307536) Homepage

          Well, here's a thought...

          Freeze the existing site. Call it archive.soylentnews.org or something.

          Make a new one. Clean slate, new maintainable everything, keeping only our logins and the general look and behavior of the site.

          So no more creaky old database or mouldering Perl, but nothing lost. Old info preserved over yonder and same community preserved over here.

          I have no idea how practical this would be, but shuttering the site would be a dead loss.

          And yeah, if I had the skills or the funds... but I don't, so... all I can do is miss y'all when we're gone. Cuz you guys really done good, and built something unique. And part of that is because growth has been so slow that we had the chance to get to know each other. We don't need millions of users; we just need us.

          --
          And there is no Alkibiades to come back and save us from ourselves.
          • (Score: 3, Funny) by rev_irreverence on Tuesday May 23 2023, @03:24AM (1 child)

            by rev_irreverence (144) on Tuesday May 23 2023, @03:24AM (#1307557)

            Maybe we could just let chatgpt post the stories. Autopilot all the way.

            • (Score: 2) by Reziac on Tuesday May 23 2023, @03:34AM

              by Reziac (2489) on Tuesday May 23 2023, @03:34AM (#1307561) Homepage

              LOL, I'm not so sure some outfits don't already do that...

              --
              And there is no Alkibiades to come back and save us from ourselves.
          • (Score: 3, Informative) by janrinok on Tuesday May 23 2023, @06:21AM (10 children)

            by janrinok (52) Subscriber Badge on Tuesday May 23 2023, @06:21AM (#1307593) Journal

            I have no idea how practical this would be

            That is the problem that we have faced for several years - it isn't practical at all.

            The number of man hours required to write any replacement code - regardless of which language we choose - would be far more than anyone is volunteering to undertake for free. We have looked at rewriting the code several times since 2014, the most recent being in 2021 when TheMightyBuzzard left. He was the last Perl programmer on the team. NCommander looked at employing outside programmers but that requires far more money than we have in our bank account.

            And even if you do rewrite from scratch, you still have to test, maintain and bug squash the results.

            It is not impossible. But finding a team that is prepared to approach the task for free has proven to be just that - impossible. I have been looking for solutions for quite some time now but the 37 day deadline has not made things any easier. We thought we were good to continue until February next year.

            So for the immediate future we are looking what is already out there that we can use to get us to where we want to be.

            • (Score: 2) by Reziac on Tuesday May 23 2023, @06:45AM

              by Reziac (2489) on Tuesday May 23 2023, @06:45AM (#1307599) Homepage

              Best of luck. If anyone can do it, you guys can. Will keep my fingers, toes, eyes, and wires crossed. :)

              --
              And there is no Alkibiades to come back and save us from ourselves.
            • (Score: 3, Interesting) by Immerman on Tuesday May 23 2023, @03:51PM (8 children)

              by Immerman (3985) on Tuesday May 23 2023, @03:51PM (#1307703)

              Yeah, I wouldn't even consider rewriting something from scratch for a dying community. But is there really no existing alternative (open source?) forum software that could be used instead? I mean, I'm sure it would look different, and not have the same mod options, but if modding is ineffective anyway, who really cares? Throw in some like/dislike buttons for the dopamine hit if you can get them and call it good.

              I'll be sad to see the site go, and personally would have no serious objection to the current forum being replaced with even a minor tweak of the sort of bog-standard forum software you see everywhere online (e.g., require (semi-automated?) mod approval for any new thread = "front page story"). I'm here for the quirky, unfocused technical community, not the forum software. And with moderation not effectively managing trolling/disinformation/etc. anyway, the fact that we have a community at all is in large part testament to the community and moderators, not the technical details of the forum.

              ... But I'm not everyone, and I can understand the nightmare it has become, and the resistance to just throwing away the quirky software that so much energy has been put into. And between the dwindling community and Slashdot having recovered from the Beta debacle, I can only imagine the business side of things has been looking pretty grim for a while now. A shame nobody could agree on at least some some basic advertising to help keep the lights on.

              At the end of the day if you're not enjoying the work, and you're not profiting from the work, you should probably seriously consider not doing the work. And if nobody else is stepping up to take over the the work... well the writing is kind of on the wall, isn't it?

              • (Score: 4, Insightful) by Common Joe on Tuesday May 23 2023, @06:56PM (7 children)

                by Common Joe (33) <{common.joe.0101} {at} {gmail.com}> on Tuesday May 23 2023, @06:56PM (#1307750) Journal

                Slashdot having recovered from the Beta debacle

                I wouldn't consider that Slashdot recovered from the debacle. Sure, they post news stories. Sometimes they are even interesting. However, hardly anyone provides thought provoking comments anymore. That's a big reason why I'm still here.

                It begs the question: is there somewhere where people have gathered that provide intelligent scientific discussion on a large variety of topics besides here? I haven't found it.

                • (Score: 1, Funny) by Anonymous Coward on Tuesday May 23 2023, @11:13PM

                  by Anonymous Coward on Tuesday May 23 2023, @11:13PM (#1307779)

                  Oh, crap! How many times have we been over the "begs the question" thing? If there was nothing else that SoylentNews was to accomplish, it was to get people to stop using the logical term incorrectly. Obvious (rebuttal) is that we failed in this.

                • (Score: 2) by Reziac on Wednesday May 24 2023, @01:04AM (5 children)

                  by Reziac (2489) on Wednesday May 24 2023, @01:04AM (#1307800) Homepage

                  Slashdot has become... sterile. Maybe because it's too big. When I wander over to moderate (cuz I seem to have perpetual mod points again) I notice that it doesn't get the firebrand on a soapbox that we still sometime get. Mostly just generic same comments we saw last week.

                  SN is a local coffeehouse.

                  Slashdot is the mall.

                  Where do you prefer to spend your leisure time??

                  --
                  And there is no Alkibiades to come back and save us from ourselves.
                  • (Score: 3, Interesting) by Common Joe on Wednesday May 24 2023, @02:06AM (3 children)

                    by Common Joe (33) <{common.joe.0101} {at} {gmail.com}> on Wednesday May 24 2023, @02:06AM (#1307807) Journal

                    I thought my answer was clear -- I prefer the local coffeehouses. I'll rephrase my question: are there any other local, good coffeehouses that I don't know about? (Because I suck at finding them.)

                    • (Score: 2) by Reziac on Wednesday May 24 2023, @02:34AM (2 children)

                      by Reziac (2489) on Wednesday May 24 2023, @02:34AM (#1307813) Homepage

                      Oh yes, clear enough -- you just stimulated my Node of Reply. :)

                      I don't know where else to find 'em either. I used to look (tho nowadays I'm far less social and more inclined to stick with what already works).

                      Well, there are all sorts of generic and specialty forums and the fediverse and all that, and they all serve an audience, but ... none hit the same sweet spot.

                      --
                      And there is no Alkibiades to come back and save us from ourselves.
                      • (Score: 3, Insightful) by Common Joe on Wednesday May 24 2023, @02:42AM (1 child)

                        by Common Joe (33) <{common.joe.0101} {at} {gmail.com}> on Wednesday May 24 2023, @02:42AM (#1307816) Journal

                        I used to look (tho nowadays I'm far less social and more inclined to stick with what already works).

                        I think someone cloned me without my knowledge -- lol. This is exactly me.

                        And I totally get the missing "the sweet spot". It's just not the same elsewhere. I like "seeing the same faces". I know what to expect from certain people and how much weight to give their comments. I'll probably wait until end of June (or even July) to start looking...

                        • (Score: 2) by Reziac on Wednesday May 24 2023, @03:10AM

                          by Reziac (2489) on Wednesday May 24 2023, @03:10AM (#1307822) Homepage

                          I'm following you around in advance. :D

                          And what you said about weighting comments. We develop that here, after a while. We know who sits at which table, and what they order, and if they leave a tip.

                          I don't intend to look for a replacement. I have enough to do without this, but... I'd rather it be SN.

                          --
                          And there is no Alkibiades to come back and save us from ourselves.
                  • (Score: 0) by Anonymous Coward on Wednesday May 24 2023, @10:28PM

                    by Anonymous Coward on Wednesday May 24 2023, @10:28PM (#1308029)

                    Slashdot has become... sterile

                    Help control the rehash problem, have your sysops spayed or neutered.

          • (Score: -1, Spam) by Anonymous Coward on Wednesday May 24 2023, @05:22AM

            by Anonymous Coward on Wednesday May 24 2023, @05:22AM (#1307828)

            Janrinok banned aristarchus to protect Runaway. Soylent News is dying because it couldn't recover from a rogue admin censoring a poor inncoent Soylentil. Soylent News is sinking like Atlantis, soon to be lost forever to protect a treasonous racist transphobe. A new site will be the same, Runaway posting violent threats and Janrinok protecting him. The stoopid Dalek will be at the new site to suck Janrinok off because his wife is dead, LOL! Janrinok supported Runaway at first because he agreed with him. Now he's too senile to know the difference and he just gets sucked off by Dalek nonstop. After his pals Dalek and Runaway get to the new site, Janrinok will ban all new users just like here. He's banned over 30 new users in the last six hours even. Oh, the huge manatees!

  • (Score: 4, Touché) by Thexalon on Tuesday May 23 2023, @12:20PM

    by Thexalon (636) on Tuesday May 23 2023, @12:20PM (#1307634)

    "Yes, please, continue to run older versions of software riddled with known security holes!" - black hat bad guys everywhere

    --
    The only thing that stops a bad guy with a compiler is a good guy with a compiler.
  • (Score: 2) by dwilson on Tuesday May 23 2023, @04:52PM

    by dwilson (2599) Subscriber Badge on Tuesday May 23 2023, @04:52PM (#1307718) Journal

    Poor advice. I prefer, 'If it ain't broke, fix it until it is'.

    All joking aside, I've found that most people don't truly understand how a thing works until they've broken it through ignorance and been forced to correct the problem. As a bonus, you're generally much better placed to fix the next problem that crops up with it, by virtue of having experience.

    --
    - D