SoylentNews is people
SoylentNews
perl code doc project
work in progress
a minor difficulty i'm having with wrapping my head around slashcode is figuring out where functions are declared. i can use a search tool like sagasu, but i've done something similar to this for php so i thought it would be a fun perl project.
objective: parse code files in a directory tree and output page with linked index of files and functions
doc.pl
#!/usr/bin/perl
print "Content-Type: text/html\n\n";
use strict;
use warnings;##########################
sub doc__main {
print "<!DOCTYPE HTML>\n";
print "<html>\n";
print "<head>\n";
print "<title>Slashcode Doc</title>\n";
print "<meta name=\"description\" content=\"\">\n";
print "<meta name=\"keywords\" content=\"\">\n";
print "<meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\">\n";
print "</head>\n";
print "<body>\n";
print "<p>blah</p>\n";
print "</body>\n";
print "</html>\n";
}##########################
sub doc__functionTree {
my($structure, $allDeclaredFunctions, $allFunctions, $allFiles) = @_;
}##########################
sub doc__recurse {
my($structure, $allDeclaredFunctions, $allFunctions, $allFiles, $allTreeItems, $caption, $type, $level, $id) = @_;
}##########################
sub doc__aboutFile {
my($structure, $allFunctions, $allFiles, $fileName) = @_;
}##########################
sub doc__aboutFunction {
my($structure, $allFunctions, $allFiles, $functionName) = @_;
}##########################
sub doc__linkFile {
my($allFiles, $fileName) = @_;
}##########################
sub doc__linkFunction {
my($allFunctions, $functionName) = @_;
}##########################
sub doc__allFiles {
my($structure) = @_;
}##########################
sub doc__allFunctions {
my($structure) = @_;
}##########################
sub doc__declaredFunctions {
my($structure) = @_;
}##########################
sub doc__loadStructure {
}##########################
sub doc__parseFile {
my($structure, $fileName) = @_;
}##########################
doc__main();
1;
Bite my shiny IRC
Won't you join us?
irc://irc.soylentnews.org/Soylent
Some have asked why we run our own servers instead of using a public one such as freenode.net. We did this to have control of the TOS, copyright, DMCA, and other legal issues. I like freenode (and their TOS) a lot, but we're building a community and we should make our own choices.
We've got a bot named Bender that monitors the newsfeed and posts announcements whenever a new article comes up.
Bender also posts the headlines to our twitter account, so feel free to follow us there for timely announcements. (Nineteen followers today - woot!)
And FrogSmash, our overlord of graphics arts is distinguishing our various communication channels slightly so that, for example, bookmarks have differently colored icons to make them more easily identifiable by eye. He's set up a test twitter account to try out new skins - check it out and send him some feedback.
Landon, our overlord of IRC, set all this up. He even set us up a link-shortener sylnt.us domain for the twitter account: that rocks! So send him some love if you see him on IRC - he's doing a bang-up job!
perl
I'm a perl noob. Hopefully if I do some journal writing on my experience it will help keep me motivated.
Got some sort of perl server configuration going. Google not very helpful since most guides are for mod_perl pre 2.0 and apache foundation docs are jibberish to me (maybe I'm just stupid).
Anyway, here's a conf that I kinda butchered up based on a bunch of different sources:
<VirtualHost *:80>
ServerName slash
DocumentRoot /var/www/slash/
Redirect 404 /favicon.ico
<Directory />
Order Deny,Allow
Deny from all
Options None
AllowOverride None
</Directory>
<Directory /var/www/slash/>
SetHandler perl-script
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
Options +ExecCGI
Order Allow,Deny
Allow from all
</Directory>
LogLevel warn
ErrorLog /var/www/log/slash/error.log
CustomLog /var/www/log/slash/access.log combined
</VirtualHost>
By the way, this is for Debian Squeeze.
My first hellow world script was also a bit more of an adventure than expected. Most tutorials leave out a header in examples.
/var/www/slash/test.pl
#!/usr/bin/perl
print "Content-Type: text/html\n\n";
use strict;
use warnings;
print "Hello world.\n";
I could (probably should) have used a text/plain mime header, but it worked nonetheless.
Also I can apparently use the following to add a path to @INC
use lib "/var/www/slash/Slash";
I downloaded the soylent/slashcode master branch from https://github.com/SoylentNews/slashcode/archive/master.zip so that I could have a squiz and see if I could be of any help with debugging etc, but although I can read some of it, I need to go to perl school before I can contribute.
My bread and butter programming languages are Delphi and PHP.
This explains a lot about the beginning of slashcode functions that aren't familiar to me:
http://stackoverflow.com/questions/17151441/perl-function-declaration
Perl does not have type signatures or formal parameters, unlike other languages like C:// C code
int add(int, int);int sum = add(1, 2);
int add(int x, int y) {
return x + y;
}Instead, the arguments are just passed as a flat list. Any type validation happens inside your code; you'll have to write this manually. You have to unpack the arglist into named variables yourself. And you don't usually predeclare your subroutines:
my $sum = add(1, 2);sub add {
my ($x, $y) = @_; # unpack arguments
return $x + $y;
}
Is it possible to do pass by reference in Perl?
http://www.perlmonks.org/?node_id=6758
Subroutines:
http://perldoc.perl.org/perlsub.html
SNQ*: Squadron of Circus Chickens or Barrel of Rabid Geese?
Betteridge be damned: which is better?.
I think I'll keep the CC Squad as protection and use the BRG as a grenade! I also have a spare Barrel of Uber Robots but I'm not sure what their stats are.
* SNQ is short for "Soylent News Question"
Salient SoylentNews Links
As the SoylentNews site has gone live, I've seen several URLs posted for access to different "areas" of the site as well as to other supporting resources. I'm using this space to collect the SoylentNews links I've found, in no particular order. Some are for historical reference, others for current access/reference.
The following links may be somewhat dated or obsolete:
Alternative URLs listed here were found at the top of http://irc.sylnt.us/
- sylnt.us - wiki
- sylnt.us - forum
- sylnt.us - code
- sylnt.us - IRC Logs
- sylnt.us - Got Bugs?
- sylnt.us - Who's Who?
- sylnt.us - Leave your IRC comments here
If you are new to IRC, a good place to start is the www.irchelp.org web site!
More #Soylent IRC-related links: NOTE: issue "/msg NickServ help" to get started.
http relay
Lately I've been working on a little tool to allow remote access to some intranet applications I've been working on. Would be interesting to see what others here thought about the concept.
The applications are normally only accessible on a LAN, with the usual NAT router to the internet.
The aim is to be able to access the applications from the internet without port forwarding in the router.
I've heard of things like BOSH (http://en.wikipedia.org/wiki/BOSH) but haven't found much in the way of specifics and I'm not sure if it does what I want.
The general idea I've been working on is to use a publicly accessible host as a relay between the client (connected to the internet) and the application server (connected to a LAN).
This is kinda how it works at the moment:
To allow remote access, a workstation on the LAN must have open a browser to a URL that uses iframe RPC to periodically poll the relay server. I've set this interval to 3 seconds, which seems OK for testing purposes (would need to be reduced for production). Every 3 seconds the LAN server sends a HTTP request (using php's fsockopen/fwrite/fgets/fclose) and the relay server responds with a list of remote client requests. Most of these responses are empty unless a remote client requests something.
From the remote client perspective, if a user opens their browser to a URL on the relay server, they would normally be presented with some kind of authentication process (I've neglected that for testing purposes) and then they would be able to click a link to access an application that would normally be restricted to the LAN. When they click that link, the relay server creates an empty request file. To respond to the LAN server with a list of requests, the relay server reads the filenames from a directory and contructs the requests list based on files with a certain filename convention (for testing i'm just using "request__0.0.0.0_blah" where 0.0.0.0 is the IP address of the remote client and blah is the raw url encoded request (special chars replaced with % codes).
So one job of the relay server is to maintain a list of remote client request files (including deleting them when the requests have been fulfilled). It would probably be best to use a simple mysql table for this, but for testing I've just used a simple text file in a location that can be written to by apache.
After saving the request, the relay server script instance initiated by the remote client doesn't die, but loops until the request file isn't empty. So while the following is going on, this instance is just looping (although it has a timeout of 5 secs).
After a remote client requests an application from the relay server and the LAN client requests the remote client requests from the relay server (asynchronously, hence the need to use a file or database) the LAN server (through the LAN client iframe and a bit of js) constructs a HTTP request and sends it to the application server (for testing purposes the RPC stub sends the request to its own server, which is processed by the application through a dispatch handler). The application response is returned by fgets call and is processed to modify hyperlinks and img sources etc to suit the relay server instead of the LAN server (still working on this bit for testing) and then posts another request to the relay server with the application page content.
The relay server then takes the page content and saves it to a text file.
The relay server script instance mentioned earlier, that is busy looping away, is checking for the existence of this page content in the request file. I tried doing this check with a call to php's filesize function, but didn't seem to work (thought maybe something to do with the writing and filesize processes being asynch but I don't know) but I found that reading the file using file_get_contents and checking if the content length is greater than zero seemed to work (though not very efficiently I'll admit).
So if the LAN server HTTP request to the relay server containing the application page content gets written to the remote client request file on the relay server, the remote client process on the relay server will read it and output it to the remote client.
If the application page content is output, or the content checking loop times out, the request file is deleted.
Except for link/img targets everything works in testing; I can request a page and it renders on the remote client browser as it would on the LAN (minus images).
Does anyone have any thoughts on this?
The code is fairly simple and short; there's a single routine on the relay server with about 150-odd lines of very sparse code, and there's a single routine on the LAN server with about 100 lines of code (will grow a bit when I get the link/img replacement and get/post param forwarding working, but not much). The application that generates the page content being relayed is thousands of lines of code but I've kept the remote stuff separate.
I'm pretty sure there are dedicated appliances that do this kind of stuff, but does anyone have any experience with them?
There's no doubt other ways to skin this cat, but I'm interested in security, simplicity and of course cost. Aspects that I liked about this approach were that I didn't have to punch a hole in the router and that the process was controllable and monitorable from the client within the LAN (every poll outputs a request status summary).
Would be interesting to find out if you think the idea is good or shit, or if there are aspects that could be improved (no doubt there are plenty). Feel free to comment or not.
Thanks to all those who made SoylentNews a reality!
edit: the setup in this case is a little different from the usual dmz/port forwarding case in that there aren't any ports exposed in the LAN router; i get through because the relay server only ever responds to outbound requests originating from the LAN server. there aren't ever any outbound requests originating from the relay server directly
Home!
So lovely to be back!!! Yes that's how it feels isn't it?
Had two UIDs on that other site, one relatively ancient forgotten one and one mostly unused as I joined the AC horde :)
But now... now home has been rebuilt! Awesome. Way back then I don't think I truly appreciated what was available --and I'm probably not the only one this applies to-- but now that we have lost it we have gained more so this time I'll try to make better use of it.
Not that I'll be prolific or anything like that but I'll scamper about once in a while *crams stuff into 255 char bio*.
Where we go from here
To everyone who contributed to first rollout, thank you! It was an amazing effort, and we couldn't have done it without you.
I've set down some notes and status, with an overview of where I see the project heading in the next few weeks. As always, we can stop and discuss if the community feels we should be moving in a different direction.
Thus begins the status:
Some have noticed that we don't have a structure or plan for development. This was *on purpose* for the duration of first release. I wanted to stay out of the developers' way and avoid anything that wasn't directly related to the rollout.
We were wildly successful, and can now proceed at a more leisurely pace. I have always intended to do development the right way - a strong foundation of tools, with people to oversee and coordinate the effort between people and other groups.
We have some overlords in place, but finding them is exceedingly difficult since I only know people from E-mail and for less than a week. For the near term, I'm the overlord of development, and I'm looking for someone to fill that spot. (And sys and style)
For this upcoming week I've told sys to take a break. Do minor bug fixes
at a leisurely pace if they feel bored, but I want people who are relaxed and refreshed. I don't want to lose people, and this has already happened.
The people who made the rollout happen are in the sys group. It's tiny (about a dozen people) and is concerned with system and server issues: bandwidth tiers, Varnish, Sphinx, linode accounts, registrars, load balancers, and so forth.
There's a much bigger group "dev" which is all the people who want to help develop code. There's some overlap, and of course everyone in sys has been doing dev for the past week. Anyone in sys is welcome to do dev work at any time, someone in dev who wants to do sys work has to be vetted.
A third group is "content": story editors, graphics, the wiki, forums, IRC and related. I'm hoping we'll have a more rich and varied landscape of content than just the news feed; for example, Landon (the overlord of IRC) wants to try weekly IRC chats with notable people, and Cactus (of the wiki) suggested the wiki could have entries for interesting discussions which are largely settled, but which keep cropping up.
A fourth group is "style": how the site is presented. CSS, layout, usability, ergonomics, advice on functionality.
The fifth group, "business", has not officially started (I have a total of three, count them thee, volunteers). This will be business-related topics such as marketing, legal, finance, [business] governance, and so on.
So for the near term, for a week or so, the editors are serving us delicious and interesting stories, while our users are getting comfortable with the system.
There's been some concern about decisions made during first rollout. I
promised that we would operate by community consensus, and I have to be looking at the big picture anyway. We can't afford to alienate anyone,
especially in these first stages.
In light of this:
1) We will use Git/GitHub for source control, since this seems to be consensus.
2) NCommander has proposed a heirarchy of development servers (three, in
addition to of the production server IIRC) for development, testing and
experimentation. We'll go with this because it's a good system that works for other projects (ie - the structure has been vetted) and it covers or surpasses suggestions from the community.
3) We will revisit the bug tracking system. Dev thinks it may be appropriate to have different systems with different intent (different systems, not different projects within one system). Dev should sort this out in the next 2 weeks or so, I'll make an executive choice if there is no general agreement. Start discussing!
4) The next dev effort will be version 2.0 of the newsfeed. Whether this is a rewrite or fixup will be based on community input. I've looked at
the perl code and NCommander's assessment that "it's not too bad" is entirely accurate for large swaths. I'm naming the effort v2.0.
As usual, if you have concerns feel free to E-mail and we'll talk.
R. Barrabas
I dreamt of Atari
I had a dream last night that I visited a school. All the classrooms were in the basement of a building...the only way to get between rooms was by boat. When I finally got to the computer room, I found my workstation. My old Atari 800 was there waiting for me. I sat down at it and immediately started working on Soylent News. This seems like some twisted metaphor.
The is second try subject line.
More text, I just want to see the error message.
Don't know how much filler is required.
https://soylentnews.org/~MichaelDavidCrawford/journal/
