SoylentNews

// ==UserScript==
// @name Soylent Upgrade
// @match http://soylentnews.org/submit.pl
// @match https://soylentnews.org/submit.pl
// @match http://soylentnews.org/submit.pl?op=viewsub&subid=*
// @match https://soylentnews.org/submit.pl?op=viewsub&subid=*
// @match http://soylentnews.org/admin.pl
// @match https://soylentnews.org/admin.pl
// @match http://soylentnews.org/admin.pl?op=edit&sid=*
// @match https://soylentnews.org/admin.pl?op=edit&sid=*
// @match http://soylentnews.org/comments.pl*
// ==/UserScript==

var simplifyChars = true;

var boxes = document.getElementsByTagName("textarea");
for (var x=0; x<boxes.length; x++)
{
    if (boxes[x].name == "introtext" || boxes[x].name == "bodytext")
    {
        var temp = boxes[x].value;
        temp = temp.replace(/<\/p><p>/g,"<\/p>\n\n<p>");
        temp = temp.replace(/<br>\s?<br>/g,"<\/p>\n\n<p>");
        temp = temp.replace(/<\/blockquote><p>/g,"<\/blockquote>\n\n<p>");
        temp = temp.replace(/<\/p><blockquote>/g,"<\/p>\n\n<blockquote>");
        temp = temp.replace(/<blockquote><div><p>/g,"<blockquote><div>\n\n<p>");
        temp = temp.replace(/<\/p><\/div><\/blockquote>/g,"<\/p>\n\n<\/div><\/blockquote>");
        temp = temp.replace(/<\/blockquote><blockquote>/g,"<\/blockquote>\n\n<blockquote>");
        temp = temp.replace(/<p class="byline">\s/i,"<p class=\"byline\">");
        temp = temp.replace(/<p>\s/g,"<p>");
        temp = temp.replace(/\s<\/p>/g,"<\/p>");
        temp = temp.replace(/<\/li><li>/g,"<\/li>\n<li>");
        temp = temp.replace(/<\/li><\/ul>/g,"<\/li>\n<\/ul>");
        temp = temp.replace(/<\/p><ul>/g,"<\/p>\n\n<ul>");
        temp = temp.replace(/  /g," ");
        if (simplifyChars)
        {
            temp = temp.replace(/\u2018/g,"'");
            temp = temp.replace(/\u2019/g,"'");
            temp = temp.replace(/\u201C/g,"\"");
            temp = temp.replace(/\u201D/g,"\"");
            temp = temp.replace(/\u2026/g,"..."); // ellipsis
        }
        boxes[x].value = temp;
        boxes[x].rows = 32;
    }
    var toolbar = document.createElement("div");
    var tempbutton = document.createElement("input");
    tempbutton.setAttribute("type","button");
    tempbutton.setAttribute("value","Blockquote");
    tempbutton.setAttribute("onmousedown","addBlockquote(document.getElementsByTagName('textarea')["+x+"]);");
    toolbar.appendChild(tempbutton);
    boxes[x].parentNode.insertBefore(toolbar, boxes[x].nextSibling);
}

var temp = document.createElement("script");
temp.appendChild(document.createTextNode("function addBlockquote(area) { var sel = getSelection(); if (sel.length != 0) { area.value = area.value.replace(sel,'<blockquote>'+sel+'<\/blockquote>'); } }"));
document.getElementsByTagName('head')[0].appendChild(temp);

function getSelection() {
    return (!!document.getSelection) ? document.getSelection() :
           (!!window.getSelection)   ? window.getSelection() :
           document.selection.createRange().text;
}

to bytram:

i've bashed together a scripty that will hopefully work as a starting point for automating some stuff on dev

exec command "~slash-test" (restricted to use by your nickserv account, and bot admins) may be run from either #dev or #test channels. at the moment it won't work cos i've already submitted the test comment (dev will return a duplication error), but its ready for preparation of test comment files similar to this one: https://github.com/crutchy-/exec-irc-bot/blob/master/scripts/slash_tests/test01

note: the parent element should contain the filename of the parent test file if applicable (leave blank or omit for root level). ie: for child of test01, put <parent>test01<parent>

source code files here:
https://github.com/crutchy-/exec-irc-bot/blob/master/scripts/slash_test.php
https://github.com/crutchy-/exec-irc-bot/blob/master/scripts/sn_lib.php

blockquote {border-left:3px solid #0F0 !important; padding-left:1em !important;}

/* Submissions */

.data .status0 {background:#FFF !important; color:#080 !important;}
.data .status0 a {color:#080 !important;}
.data .status0 a:visited {color:#0A0 !important;}
.data .status0 a:hover {color:#0C0 !important;}
.data .status1 {background:#800 !important;}
.data .status2 {background:#256625 !important;}

NOTE: This is a work-in-progress; read at your own risk/confusion. It is an attempt to gather together bookmarks, tabs, and information pertaining to Unicode, UTF-8, HTML, and 'characters'.

It would seem to be a simple enough question to answer, but things are not always as they seem:

What characters should SoylentNews support?

Motivation: as many of you are aware, one of the early improvements that SoylentNews made to its base source code was to support Unicode characters. (Thanks to the heroic efforts of The Mighty Buzzard.) The underlying code only supported ASCII (American Standard Code for Information Interchange) characters. Which was just fine for as far as it went. It just didn't go far enough for us...

I took on the task of testing our implementation of UTF-8 support. Little did I know what I was getting into! It has been a fascinating journey, indeed!

What is Unicode?

This is taken from What is Unicode?:

Fundamentally, computers just deal with numbers. They store letters and other characters by assigning a number for each one. Before Unicode was invented, there were hundreds of different encoding systems for assigning these numbers. No single encoding could contain enough characters: for example, the European Union alone requires several different encodings to cover all its languages. Even for a single language like English no single encoding was adequate for all the letters, punctuation, and technical symbols in common use.

These encoding systems also conflict with one another. That is, two encodings can use the same number for two different characters, or use different numbers for the same character. Any given computer (especially servers) needs to support many different encodings; yet whenever data is passed between different encodings or platforms, that data always runs the risk of corruption.

Unicode is changing all that!

Unicode provides a unique number for every character, no matter what the platform, no matter what the program, no matter what the language. The Unicode Standard has been adopted by such industry leaders as Apple, HP, IBM, JustSystems, Microsoft, Oracle, SAP, Sun, Sybase, Unisys and many others. Unicode is required by modern standards such as XML, Java, ECMAScript (JavaScript), LDAP, CORBA 3.0, WML, etc., and is the official way to implement ISO/IEC 10646. It is supported in many operating systems, all modern browsers, and many other products. The emergence of the Unicode Standard, and the availability of tools supporting it, are among the most significant recent global software technology trends.

Here is an excerpt from Wikipedia's entry for Unicode:

Unicode has the explicit aim of transcending the limitations of traditional character encodings, such as those defined by the ISO 8859 standard, which find wide usage in various countries of the world but remain largely incompatible with each other. Many traditional character encodings share a common problem in that they allow bilingual computer processing (usually using Latin characters and the local script), but not multilingual computer processing (computer processing of arbitrary scripts mixed with each other).

Unicode, in intent, encodes the underlying characters—graphemes and grapheme-like units—rather than the variant glyphs (renderings) for such characters. ...

In text processing, Unicode takes the role of providing a unique code point—a number, not a glyph—for each character. In other words, Unicode represents a character in an abstract way and leaves the visual rendering (size, shape, font, or style) to other software, such as a web browser or word processor.

A little more background: There are certain code points in Unicode that are of questionable value in the context of a web page; further, there are code points which are defined to be invalid! And then, just to make things even more interesting, I found a list of invalid characters in an HTML document:

Illegal characters

HTML forbids[6] the use of the characters with Universal Character Set/Unicode code points (in decimal form, preceded by x in hexadecimal form)

• 0 to 31, except 9, 10, and 13 (C0 control characters)
• 127 (DEL character)
• 128 to 159 (x80 – x9F, C1 control characters)
• 55296 to 57343 (xD800 – xDFFF, the UTF-16 surrogate halves)

The Unicode standard also forbids:

• 65534 and 65535 (xFFFE – xFFFF), non-characters, related to xFEFF, the byte order mark.

UTF-8; Unicode Transfer Format - 8-bit

Though there are several means by which Unicode characters can be transmitted between contexts, one of the most popular is UTF-8, which is what was chosen for use in SoylentNews.

SoylentNews:

What you see from our site mostly comes via a browser (though we also support Gopher and NNTP; you can have stories e-mailed to you; and we also have an RSS/Atom feeds... wow!)

Our site currently formats web pages as HTML 4.01; here's a representative DOCTYPE:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
            "http://www.w3.org/TR/html4/strict.dtd">

At some point in the future we may want to directly support HTML5; ideally nothing should preclude or complicate that effort.

See also:

• What is Unicode?,
• Unicode,
• Unicode and HTML,
• C0 and C1 control codes,
• [20] SGML Declaration of HTML 4,
• Characters not Suitable for Use With Markup,
• HTML, XHTML, XML and Control Codes (Updated 2003-09-06 15:10) Answers the question: How do I handle control codes (ie. the 'C0' U+0000-U+001F and 'C1' U+007F-U+009F ranges) in XML, XHTML and HTML?
• About the Unicode Character Database,
• Unicode Character Database,
• Index of /Public/UCD/latest/ucd,
• http://www.unicode.org/Public/UCD/latest/ReadMe.txt,
• Unicode 7.0.0 (Released: 2014 June 16),
• Articles, best practices & tutorials,
• Serving HTML & XHTML

Obviously, we need not support the invalid code points. (Enumerate them here).

Unicode and UTF-8

So Unicode is a collection of mappings of code-points (numbers) to 'characters'; UTF-8 is a Unicode Transformation Format, 8-bit, used to transmit/encode Unicode code points.

To be continued...

Sigh, with it becoming harder and harder to find decent single player games, as i am not part of the "dudebro" demographic and therefor do not care for your "Medal Of Dooky: Halo Of Killzone Gears Edition" style of games I decided to jump in and try one that looked the least "dude bro"ish and that game is....War Thunder.

I have to give the designers of the game credit, as nearly ever other "FTP" game I've ever played is either so grindtastic its insane or so badly pay to win its not funny but at least in these early tier 1 games? Its really not, hell you can take the default tank (that you get free repairs on forever so you can be as nutty as you like with it, nice) and actually start getting kills in the first few minutes of the first game, its really all about keeping your head on a swivel and remembering to watch your reload times. Nothing will give you a smile quicker than somebody coming to strafe the tank formation you are in and you blow its wing off with the 20mm of your Panzer II lol.

So if any of you are getting sick of the usual fare and want something a little different? Do NOT be put off by the fact its planes and tanks (and soon to be ships) as its beyond easy to drive with a standard keyboard and mouse. If you can play an FPS? You can play War Thunder. Maybe if a few guys here give it a spin and like it we can get a team together, having :"Team Soybeans" roll the tanks while "Run To The Hills" by Iron Maiden plays? That would be a blast.

currently developing a sort of api for the exec irc bot.

bit of background: exec is run as a single php script process in a terminal, which connects to the soylent irc server, but when commands are triggered from the irc channels, the bot spawns child processes to service the commands. command aliases are mapped to shell commands using templates to substitute data from irc (such as nick, channel, message, etc). child processes are usually passed data using command line arguments, and communication to/from child processes and the bot process is facilitated through child process stdin and stdout pipes.
eg: typing "~rainbow ciri is a butthole" triggers the shell command: php scripts/rainbow.php 'ciri is a butthole'

the bot creates a named pipe and dumps a bunch of data to it in response to events, such as when a child process is started, when a message is received from a child process stdout pipe, when a message is received from the irc server, etc.
the named pipe can be read by something simple like cat, but i've written another php script to read it. this reader script also has a tcp socket server which allows yet another program to access the data stream (remotely even), such as telnet.
this is fine for just reading data, but i wanted to be able to command the bot remotely using this api. this is achieved by starting the reader script as a child process of the bot, which then offers the reader script the same methods taken advantage of normal scripts through stdin and stdout. a couple of specific stdout handlers were added to the bot script for acquiring specific data, but the reader also has direct access to irc and other functionality using existing stdout handlers (/IRC and /INTERNAL being the main handlers, used to output a raw message to the irc socket and perform an internal command respectively).
with two-way communication established between a tcp socket in the reader script and the main bot process, with event data streaming, i wanted to make a pretty gui application that i could see what is going on, since the bot terminal can be a bit crazy (each irc message can result in output filling half the terminal or more, due to internal events and data messages).
my tool of choice is delphi 7 pro, so i've made a little app called 'execstat' that connects to the reader socket server and reads the data streaming from it and enables commands to be sent. its pretty basic at this stage, but hope to develop it into something that i can use to easily see which scripts are running, which buckets are being get/set, which channels the bot is in, which commands are being triggered by who and where, etc. also important will be the ability to trigger kill commands for scripts, since currently to kill a script i have to go to a private channel, get a list of running processes (using the ~ps alias) and enter the pid in a ~kill alias command. not hard, but would be nice if i could just click a button in a gui. lazy++

the socket in the delphi app reads in its own thread, and uses a synchronized event handler to give data to the main application thread.

my current work revolves around rolling my own unserializer classes to parse the php serialized arrays, which is necessary to be able to get anything useful from the data streaming into the socket.

this whole setup is probably way overcomplicated and rediculous, but i enjoy working on it. moar_complexity++

per previous journal i set up a simple channel relay feed to enable status messages for ncommander's nethack stream, but since there has been a little bit of talk about other games (including dungeon crawl stone soup) i turned the feed code into something more generic so that other feeds could be added easily. now there's feeds for the rodney bot from #nethack (freenode) to #nethack (soylent), as well as feeds for a couple of bots from ##crawl (freenode) to #crawl (soylent). the code to dispatch messages containing keywords has also been made generic, so that if ncommander plays again his deaths can be easily dispatched to #soylent for our amusement :p
code for this feature can be found here: https://github.com/crutchy-/exec-irc-bot/blob/master/scripts/activity.php#L86-L115

another feature that was briefly mentioned in my last journal has been developed further to enable limited two way communication between someone in a soylent irc channel and someone in a freenode irc channel.
to send a message to a channel on the freenode irc network (that an exec minion resides in with forwarding enabled) an authorized user on the soylent irc network in a channel with exec can type "#freenode_channel > message" and the message will be relayed. if you're in the freenode landing channel on the soylent irc network (#freenode) you'll be able to see any replies pop up in there, but it can easily get hidden amongst all the traffic from other channels, so to enable a 10 minute automated relay to a different soylent irc channel, type "#freenode_channel > freenode_nick, message" and for the next 10 minutes anything that freenode_nick says in #freenode_channel will be relayed to the originating soylent channel. only messages prefixed with "#freenode_channel >" will be relayed from soylent to freenode so soylentils can still talk without anyone on freenode hearing, but using the prefix on the soylent end (not required on freenode end) enables two-way communication. the relay lasts for 10 minutes, but can be easily renewed as many times as you want by typing another message (on soylent) in the form "#freenode_channel > freenode_nick, message". if you want to see a list of currently active relays, type ".relays" and it will list freenode_nick, freenode_channel, soylent channel and time remaining (in minutes). use of this feature is limited to a list of nickserv accounts.
code for this feature can be found here: https://github.com/crutchy-/exec-irc-bot/blob/master/scripts/activity.php#L133-L213

re-enabled the ~submit alias so that instead of just pasting a link to irc, you can also (with smallest amount of additional effort) share it with SN users not on irc. syntax is really simple: ~submit %url%
just submits the description or og:description meta tag content along with the link, so makes for a pretty lame submission but its better than nothing at all (editors can always reject it if it sucks too much)

during NCommander's NetHack thing i tweaked the freenode relay script to output messages by the Rodney irc bot in the freenode #nethack channel to our #soylent channel when NCommander died in the game. also made it so that anything Rodney said was output to our local #nethack channel so that NetHack fans can see what's going on without connecting to freenode directly.

developed some macro functionality to enable authorized irc users to shorten otherwise long exec commands into shorter equivalents. came in handy for simplifying control of the freenode relay bot.
also made a compound macro (using exec's internal_macro lib function) to make it easy to talk through the relay bot from the SN irc channels to freenode channels.

we get a bit of spam occasionally in the wiki, and chromas is diligently blanking it out (despite not having authority to delete articles) as a means of spam control. a script has been developed to make it a little easier for him (and a couple of others in a list of authorized nickserv accounts). a spam page in the wiki can now be cleared with ".spamctl %page title%"

http://sylnt.us/exec
http://sylnt.us/execsrc

broke out ircd work into a separate github repo. currently working on ability to connect multiple times to a network with the same nick. idea is that if there are a few servers on the network you could connect to each of them, and then if one disconnects you can still use the other (redundant) connections. authentication of subsequent connections to the same nick is done using a connection id appended to the username field (username sent out doesn't include the connection id though).
the server is at a point where i can connect to it with hexchat a couple of times (server spits out a connection id on first connect, and i just paste that to the end of the username in second connection). i can then join a channel from either connection, which automatically joins for the other connection (weird to watch). then i can chat and the chat appears on both connections. its pretty funky to play with.
if the project goes anywhere (unlikely) it might be ideal to have a client that handles the redundant connections for you automatically rather than having to establish separate connections manually and have each one take up room in the server list. netsplits could be a thing of the past.
prolly will try to make it ts6 and rfc compatible, but still early stages.

https://github.com/crutchy-/ircd

A day that will live in e-fame-y.