SoylentNews

As we get full swing into the of the next development cycle of the site, I wanted to lay out our plans for community discussion, and help set a path out for where we are going from here. As things sit right now, here's *roughly* what we're expecting in the next site update.

Rehash 15.04 - Tentative Changelog

• Migration to Apache 2.2/mod_perl 2 (complete, still bug flushing)
• Removal of all MyISAM tables in the backend (complete)
• Migration to MySQL Cluster (implemented)
• SSL by default (enabled on dev)
• Nexus support
• Rewritten site search engine, replacing MySQL FULLTEXT searches with Sphinx (in progress, mostly done)
• Private Messaging
• Inline Reply and Moderation*
• IPv6 support
• Another large chunk of dead code removal

* - if user has JavaScript enabled, non-JS users will get the current behavior.

From a user perspective, the most visible changes are just more streamlined site performance. Inline reply/moderation is something that was by far the most common request the last time we asked for feedback on the subject, and I am hoping to have it implemented with this release. (Fancy URLs, the other largely requested feature will require consider effort to find and fix URLs throughout the codebase; I do not have an ETA on this feature, aside from the fact its on our roadmap).

Migrating from standard MySQL to cluster should help us considerably keep the site up for longer periods of times; we have previously experimented with other MySQL replication solutions, and found them all lacking, especially in cases of master failover. With this update to Rehash, we can take a database server offline and not have the site come crashing down, or require exceedingly large amounts of effort to replicate and reconfigure the front-end servers.

While this covers here and now, I've long liked to have plans 6 months to a year out, and post them for community feedback so everyone is in the loop on both what we're doing, and why we're doing it.

The Next Six Months

Looking ahead, we'd like to be able to expand both our potential userbase, by allowing user-created nexuses, expanding site usability for people who do not speak English as a primary language, and allowing for the possibility of non-English spin-offs of SoylentNews, as well as continuing our ongoing work on rewriting and improving the backend of the site to both increase usability, and decrease maintenance burden. As such, I see our next major update focusing increasing our internationalization and localization abilities.

Now obviously, these are broad objectives, and there will be the usual slew of tweaks and upgrades as we move forward, so don't consider these lists to be definite and final. Furthermore, if there is pushback on the direction we're taking, I hope our previous history will show we'll sit down and work to revise our plans.

Rehash 15.06 - Tentative Changelog

• Isolation of language strings from templates to .po files, solicit community efforts to translate the site into various languages*
• Incorporate translations into Rehash, allowing a user to use the site UI in their native language
• Define and finalize plans for user-created nexuses

* - our initial focus is going to be on left-to-right languages. While RTL language support is something I'd like to support, this is likely going to require a massive amount of effort beyond just getting the site translated, so unless a large group of folks step forward to help build say an Arabic or Hebrew version of SN, RTL support is not super high on the priorities list.

Non-English speakers are likely familiar with the two major spin-offs of the other site, specifically Barra Punto and slashdot.jp

Both of these sites are running on rather dated versions of slash, and lack much of the port and polish that has gone into the back end since they were launched. We feel that we can do better, and want to give the non-English users of our site a chance to help build better news portals. We've already have significant interest in making the site available in Spanish, and I met a bunch of Polish SN users when I was in Warsaw last summer; I want to provide the community with tools to take what we've done and take it further.

Between now, and when we are technically able to, we hope to find volunteers who would be willing to edit and manage a non-English version of SoylentNews, and thus allow them to "hit the ground running" so to speak, once the site upgrade launches.

On user-created nexuses, as I've indicated before, this essentially will be our version of subreddits, allowing any topic to be discussed on SN. Prior to implementation, we, as a community, need to define a formal code of conduct, as well as determining what, if any, monetization should be done; while we can cover our hosting costs from the revenue brought in from subscriptions, I'd like to eventually get to the point that we can hire both full-time developers and editors. While this may still be years out if ever, being both fully self-sufficient, and being able to cover development and sysadmin costs (vs. being 100% dependent on volunteers) is an important step (though this is a discussion for a future article).

Moving on, let's take a look at where I'd like to see us by August ...

Rehash 15.08

• Implementation of user-created nexuses
• Assuming sufficient volunteers, launch of non-English SN in at least one language
• Defined community-governance of content on SN
• Initial port to PostgreSQL

Right now, the staff directly handle both content, and handling of users issues. For the most part, I'd like to think this works relatively well, but once we have users who can manage their own parts of SN, we need to make sure we have mechanisms in place to handle user grievances. While I'm unsure how much of our community uses Reddit, many are at least passingly familiar with the some of the drama that has come out of various subreddits, and the lack of transparency from the admins.

By having defined mechanisms of governance, this will allow the admins here to intervene should it be necessary, and allowing disputes between nexus admins and their users to be resolved with a minimum of drama and such, as well as making sure we don't compromise the values on which this site was originally founded.

On a technical level, we also are looking at moving to postgreSQL in the long term. While it may seem odd that we're doing two rounds of major database work, migration to MySQL cluster merely helps handle availability issues, aside from removing FULLTEXT searches, it has not been necessary to rewrite much code to allow the site to operate on a cluster. Migration to MySQL cluster is more an increment improvement then a massive rewrite of the guts of the site

For those unfamiliar with MySQL, it lacks many of the features that are present in more robust databases. Its not uncommon to have massive JOINs spanning 3-4 tables in the current code base, or large amounts of database logic written in Perl due to limitations in MySQL views, triggers, and stored procedures. By migrating to a more advanced database, we hope to drastically reduce (perhaps by half) the amount of code present in the front end and reducing our long-term maintenance burden. Obviously, until we start the effort to port the back end the true difficulty of this will remain unknown and it may well be, once we dig deeper into it, that the cost of migration would outweigh any benefits.

In Closing
Since we went live a year ago, SoylentNews has been, and continues to be an adventure. I'm committed to helping us reach the goals set out in the manifesto, constantly placing the community first, and allowing this site to grow and thrive. Your feedback is extremely important to know if we're going the right way, hence why I'm laying down where I want to go now, and then seeing if everyone feels like its a good move. If not, I'll go back to the drawing board, and try again. We wouldn't be here without you guys, and we're not going to forsake the folks that got us this far.

Until next time,
NCommander