SoylentNews

So, on a less serious topic than incorporation, I wanted to open the floor on a possible new feature to Slashcode; the possibility of reading and posting via a standard newsreader. This idea has gnawed on my brain since GoLive, so a few weeks ago, I started coding a proof-of-concept in my spare time to see if the idea is even feasible.

The answer: Yes, it is.
I'm mostly interested in seeing if there's enough people who would like this feature to develop it fully. As of right now, this is a single-shot export of the site, but it would be relatively easy to expand (at least in terms of SN->NetNews. Full bidirectionality is more difficult). For more information (as well as access to the proof-of-concept), read more past the break.Just to prove this actually exists:


Now, obviously, this isn't a new idea; NNTP (Network News Transfer Protocol) access for the other site was promised WAY back in 2000-ish, but I wanted to see if it could reasonably be done. Because of the quirks (and headache) of running INN (InterNet News), combined with the fact that most discussion forums these days are flat vs. threaded, very few (if any) forums offer access by NNTP. In addition, given our community is very tech-savy, I could see a fair number of users wanting a way to participate in SN without even using their web browser (the NNTP feed looks good with a newsreader on my phone).

These two aspects led me to spend a few weekends reading through RFC 1036, and learning the ins and outs of how NetNews/USENET works, and seeing how it would be possible to interface slashcode to an instance of INN. (I didn't really want to implement an NNTP server that read against our database). I discovered the "easiest" solution was to make slashcode appear as a remote news server accessible via UUCP (Unix-To-Unix Copy), which meant it was simply a matter of writing out a massive spool file, then feeding it into INN with rnews.

After a fair bit of hacking, swearing at perl, and other fun, I was successfully able to export both stories and comments into INN. This opens some interesting possibilities in terms of accessibility to the site. Assuming I continue work on this, it would allow mirroring SoylentNews across the entirety of USENET. From the progress I made thus far, I do think full bi-directionality is indeed possible (with "creative" use of moderated groups on the INN side). There are some headaches, such as some newsreaders (like Thunderbird) handle cancels and supersedes headers poorly (required for times we edit articles), but I do believe the most basic problems are surmountable.

Now, obviously, being a proof of concept, some caveats apply:

• The current setup is a single-shot import of the database into INN, and will not be updated regularly
• The main page is read-only for obvious reasons
• No in-line quoting
• • This is something of an annoying issue as slashcode and traditional USENET quoting styles don't exactly mess. I can modify the export to include one or two levels of quoting on each post, which should help with usability, but this isn't an ideal solution. If I do this, the result will be much closer to email quoting than USENET quoting
  • I'm open to discussion on how best to fix this (if at all)
• A couple of posts got mangled through the convert, especially ones that have HTML encodes, or unicode
• I reused the email generator to format both articles and comments, which means that they render "weirdly" when spooled out (they're rendered as spaces instead of ">" characters

All of the above should be considered pre-alpha quality, more like a whiteboard of insanity. If, despite all these caveats, you'd still like to check out the spool for yourself, point your newsreader at nntp-test.soylentnews.org. There are three groups on the server, soylentnews.discuss, just for general discussion for folks who come by, a test group for testing posting, and the spool itself at soylentnews.test.mainpage.

I look forward to hearing your thoughts and comments below.

[UPDATE: Corrected typos and added links]