SoylentNews is people
SoylentNews
posted by
janrinok
on Wednesday March 12 2014, @11:33PM
from the I-was-doing-OK-until-it-mentioned-perl dept.
from the I-was-doing-OK-until-it-mentioned-perl dept.
xyzzyyzzyx writes:
"Avantslash is touting a user hosted perl script that, if paired with any web browser with JavaScript, promises to shave crucial bytes off of the standard Slash-based experience, one of which is our very own SoylentNews. Audiences include those with very limited bandwidth, such as those in developing countries with only 2G mobile access or dialup."
This discussion has been archived.
No new comments can be posted.
Lowering Bandwidth Costs for Slash-based Sites
|
Log In/Create an Account
| Top
| 58 comments
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
... eighty years later he could still recall with the young pang of his
original joy his falling in love with Ada.
-- Nabokov
(Score: 5, Funny) by grub on Wednesday March 12 2014, @11:45PM
Just send out a SHA hash of the front page.
For example "1ad65bfb0225ec69e82d5a632da603b2cd4ac77c" uses of far less data than does all those wordy headlines and summaries you currently use! Anyone who can't figure out how to back the page from the hash should just go back to that other site!
Trolling is a art,
(Score: 5, Funny) by stderr on Wednesday March 12 2014, @11:54PM
Lots of pages share the same SHA hash, so you wouldn't know which one is the right one...
And that's the ONLY reason we're not just sending the SHA hash already.
alias sudo="echo make it yourself #" #
(Score: 5, Funny) by jt on Thursday March 13 2014, @02:05AM
It's OK if you can't tell which page is which among those with identical SHA hash over at Slashdot; they'll be dupes anyway.
(Score: 1) by NezSez on Thursday March 13 2014, @04:44AM
#think 50's era comic book
"Shazam!"; now I'll save the kitty in the tree!
#think 2014 metro
"SHACollision!"; now find that story about the new Cosmos series b**aatch!
No Sig to see here, move along, move along...
(Score: 4, Interesting) by GungnirSniper on Thursday March 13 2014, @12:01AM
Why not just merge this into the main Slashcode?
It's a cool project, thank you to the Devs and for GPLing it.
Tips for better submissions to help our site grow. [soylentnews.org]
(Score: 5, Informative) by hankwang on Thursday March 13 2014, @12:17AM
"Why not just merge this into the main Slashcode?"
Developer here. Although it's perl, I'm afraid that trying to integrate it with slash is doomed. It won't scale with the kind of load on SN or /. . And frankly, parsing html code with tens of regular expressions is not what you'd do if it runs on the same server as SN. (wouldn't do it anyway if I had to write it from scratch today, but this is how it evolved over 10 years, and it does the job)
But the UI and lightweight html could serve as inspiration for a mobile engine for SN.
Avantslash: SoylentNews for mobile [avantslash.org]
(Score: 2, Interesting) by goodie on Thursday March 13 2014, @01:52AM
THIS! I think that if this proves as good as it seems (can't wait to try it on my iphone during my commute tomorrow) we should find a way to "integrate" it with the SN codebase. After all what you do (if I understand it well) is reformat the info to fit a mobile device. There's no reason that we can't do that straight from the SN source without having to resort to the technique you guys have to use (which is normal on your side of the fence). Heck, in the future, we could have a simple cookie storing the mobile or desktop version of the site for viewing and redirect/pick a css based on that. I don't know if it just changes the look or involves different data as well for example.
But I do read SN a lot on my iphone when I wait for appointments, meetings, etc. so it would be great to have this integrated into the SN codebase I think. Plus we may find that there is a lot more demand than we though for something like this if we can track the pages served by this "module"/functionality.
And dammit I'm out mod points :/. But this is neat guys!
(Score: 4, Informative) by hankwang on Thursday March 13 2014, @07:27AM
"There's no reason that we can't do that straight from the SN source"
Apparently, slashcode is rather convoluted. I think Mattie.p stated that it is difficult to implement skinning. I had a look at slashcode and it seems that there are many, many layers between parsing the cgi parameters and generating output. I couldn't figure out the code path from grepping the code. It looks like making a truly mobile layout, not just applying different css, requires changes all over the code base.
I suspect that slashdot dragging their feet all those years with a mobile version has to do with the difficulty of implementing it. And that was with the original author, CmdrTaco/Rob Malda, on the team.
Avantslash: SoylentNews for mobile [avantslash.org]
(Score: 1) by goodie on Thursday March 13 2014, @12:58PM
I see... Maybe there is a way for SN to set up something on the side that consumes the data and skins separately from the main code while the mobile version only allows reading articles and comments? I admit that i have never looked at the SN codebase so if is all Perl from the late 90's it may not really have an api that we can call so to speak...
Hey if we move toward a complete rewrite maybe this could be a good starting point to slowly build the new code up...
(Score: 2) by GungnirSniper on Friday March 14 2014, @12:15AM
Bryan's Pipedot is a completely new, modern rewrite of what Slashcode does [pipedot.org] but I don't believe it uses Perl. Once that is more mature, it may make sense for that code to be the base of a new SN (or whatever name we use).
Tips for better submissions to help our site grow. [soylentnews.org]
(Score: 3, Interesting) by evilviper on Thursday March 13 2014, @06:17AM
Because it's essentially a proxy... Having the ad-blocking, excess-markup-stripping, and caching proxy (ala Privoxy) on the server-side (instead of somewhere on the user premises) at least partially defeats the very purpose...
And in that regard, I'd say the RSS feed should be vastly more bandwidth-light for the stories and links. And for the comments, you can always check-mark "Simple Design" and "Low Bandwidth" in your user preferences.
I don't mind the look, for a mobile web site, but it's a bit feature-bare when it comes to viewing, navigating, and leaving comments.
Hydrogen cyanide is a delicious and necessary part of the human diet.
(Score: 1) by Hell_Rok on Thursday March 13 2014, @12:02AM
I imagine you could do something similar with a browser extension.
You could detect the website the user is trying to go to, check if it has some sort of json/xml/etc api and display the data using local css and javascript.
(Score: 3, Insightful) by stderr on Thursday March 13 2014, @12:15AM
Browser extensions and mobile platforms don't always mix very well.
alias sudo="echo make it yourself #" #
(Score: 3, Informative) by Geotti on Thursday March 13 2014, @12:59AM
Sometimes, though, they do [saurik.com].
(Score: 0) by Anonymous Coward on Thursday March 13 2014, @06:25PM
It's only part about look and usability. A browser extension wouldn't address the part about slashing mobile download size. (and would load even slower with the extra processing being done locally)
(Score: 2) by stderr on Thursday March 13 2014, @12:12AM
janrinok, care to explain why you think it's a problem the script is written in perl?
alias sudo="echo make it yourself #" #
(Score: 2) by The Mighty Buzzard on Thursday March 13 2014, @03:08AM
My rights don't end where your fear begins.
(Score: 1) by Twike on Thursday March 13 2014, @05:21AM
Maybe janrinok remembers the old joke, executable line noise.
(Score: 2, Informative) by Anonymous Coward on Thursday March 13 2014, @09:41AM
Well, I think others have already answered this. Firstly, the Dept field tends to be lighthearted unless the topic for discussion is grave and deserving of a more serious approach. In this case, I chose the lighthearted feel. Secondly, Having spent everyday since we started slashcott providing support of some description to the guys doing the hard work, I am only too aware of the problems caused by the rats' nest of perl code that is at the heart of this project. Finally, going back and looking at perl after many years away from it - and having discovered Python in the meantime - I am amazed that I ever managed to write anything that was maintainable all those years ago. I know all the retorts about anyone can write bad code in any language, but reading perl does not feel as natural as reading many other languages, despite its obvious power and the large treasure trove that is CPAN. If it is your favourite language, good on you, but don't take offence at my poor attempt at a light-hearted quip.
(Score: 2) by janrinok on Thursday March 13 2014, @09:46AM
For those that haven't yet worked it out - I am the AC that replied above. Must have been finger trouble on my part to end up as AC!
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 2) by hankwang on Thursday March 13 2014, @05:57PM
"don't take offence at my poor attempt at a light-hearted quip."
No offence taken. I wrote most of the code and I'm in a hate-love relation with perl myself. Python is nice, but much slower to run as CGI.
Avantslash: SoylentNews for mobile [avantslash.org]
(Score: 5, Informative) by everdred on Thursday March 13 2014, @12:12AM
You can play with it here: http://soylitenews.org/ [soylitenews.org]
(Score: 1, Insightful) by Anonymous Coward on Thursday March 13 2014, @12:39AM
I actually like your comment layouts better than the SoylentNews version. It lets you see more postings by default instead of having to click the links to view each of them. Pick this format up SoylentNews!!!!!
(Score: 3, Informative) by everdred on Thursday March 13 2014, @12:49AM
(Score: 2) by Reziac on Thursday March 13 2014, @03:20AM
Nice to have if the need arises, thanks!
And there is no Alkibiades to come back and save us from ourselves.
(Score: 3, Informative) by drgibbon on Thursday March 13 2014, @05:53AM
It already exists. Just register an account and change your display mode setting [soylentnews.org] to "Nested". But yeah, I think nested would be a better default.
Certified Soylent Fresh!
(Score: 1, Interesting) by Anonymous Coward on Thursday March 13 2014, @09:36PM
I always use nested too (and browse at -1 for what it's worth).
(Score: 1) by Yog-Yogguth on Friday March 14 2014, @08:10AM
Same here, it's the only true way :)
Bite harder Ouroboros, bite! tails.boum.org/ linux USB CD secure desktop IRC *crypt tor (not endorsements (XKeyScore))
(Score: 2) by Nerdfest on Thursday March 13 2014, @12:48AM
Thanks, it's quite nice.
(Score: 2, Insightful) by Ken_g6 on Thursday March 13 2014, @12:51AM
Hm, haven't I seen this before? A site that leaves a huge blank space at the side of the screen? Either shows full comments or hides them completely, but doesn't show comment headlines?
Fsck Soylent Beta!
(Score: 1) by zafiro17 on Thursday March 13 2014, @12:21PM
Many, many thanks! I hope you leave the site up - it's very useful. I happen to like the layout better than I do the original, and I'm hugely grateful for the savings in bandwidth. This is going to be the first page I visit, and I'll go to Soy directly if I need to comment.
Dad always thought laughter was the best medicine, which I guess is why several of us died of tuberculosis - Jack Handey
(Score: 0) by Anonymous Coward on Thursday March 13 2014, @12:18AM
How about a system where content served from SN can be mirrored from user PCs? A browser plugin or program for which can receive data from SoylentNews and distribute it to other PCs via http in the same way that BT works?
Granted, some files will be updating every few minutes.. but still, for older stories and cached files it may be be beneficial.
(Score: 5, Interesting) by Appalbarry on Thursday March 13 2014, @01:00AM
First, the project web site actually describes it WAY WAY better than the summary:
NOW I'm interested. Don't just tell me how it works, tell me what it does!
Second: is this a Slashvertisement?
Third: Or is it a Soyvertisement?
(Score: 2) by clone141166 on Thursday March 13 2014, @01:23AM
I liked that the primary example screenshot of SoylentNews shown was one of McGrew's comments paying out /. users for being "clowns". :)
Also, isn't there an FDA requirement that Soyvertisements contain nutritional information?
(Score: 4, Funny) by bill_mcgonigle on Thursday March 13 2014, @03:31AM
Also, isn't there an FDA requirement that Soyvertisements contain nutritional information?
Soyvertisements contain the electrolytes readers crave.
(Score: 2) by Foobar Bazbot on Thursday March 13 2014, @03:04AM
No,it's a Soyvertiselent.
(Score: 3, Funny) by hankwang on Thursday March 13 2014, @06:17AM
"Second: is this a Slashvertisement?
Third: Or is it a Soyvertisement?"
Only if I had submitted the story myself, which I didn't. But my sig is.
Avantslash: SoylentNews for mobile [avantslash.org]
(Score: -1, Troll) by Anonymous Coward on Thursday March 13 2014, @01:03AM
This is a teaser, that's what this is. The office at this end is still busy, and can't write proper slash yet (no, not that kind of slash, rather the other kind). However, stuff is cu^Homing very soon!
Right over there, in front of the Soylent office heart-shaped bed, stand LaminatorX and mattie_p. They're wearing leather loincloths, and nothing else. There is a lot of muscular physique on show. What are they going to do? I'm glad you asked! They'll do all kinds of things, and we'll find out what. But only after there's more time on this end. Stay tuned!
(Score: 2) by mattie_p on Thursday March 13 2014, @01:40AM
I wish I could pull off the leather loincloth look. Mrs. mattie_p might like it. Then again, she might not.
(Score: 1) by http on Thursday March 13 2014, @03:09AM
How much bandwidth does it save?
I browse at -1 when I have mod points. It's unsettling.
(Score: 0) by Anonymous Coward on Thursday March 13 2014, @03:23AM
RTFA, much? It's in there.
(Score: 3, Informative) by TheLink on Thursday March 13 2014, @03:50AM
AND how much does it really save if you already have compression on the cache (or web server if you don't use a cache)?
https://www.varnish-cache.org/docs/3.0/tutorial/co mpression.html [varnish-cache.org]
https://en.wikipedia.org/wiki/HTTP_compression [wikipedia.org]
I believe most mobile browsers support gzip nowadays.
High text sites like SN compress really well. An intelligent cache would only have to do the compression once whenever the page changes.
(Score: 2) by everdred on Thursday March 13 2014, @10:18PM
I can actually answer this from experience, having run a public instance of Avantslash for about a day now. The average pageview weighs in at just under 5 KB.
(Score: 1) by hankwang on Friday March 14 2014, @10:37PM
5 kB is quite low. Is your server doing gzip/deflate compression?
Avantslash: SoylentNews for mobile [avantslash.org]
(Score: 1) by everdred on Wednesday March 19 2014, @03:29PM
gzip is enabled. It seems to compress pages down about 60%, but they're rather small to begin with.
(Thank you for the software, by the way!)
(Score: 1, Troll) by Khyber on Thursday March 13 2014, @04:17AM
http://tools.pingdom.com/fpt/#!/b5U5S6/soylentnews .org [pingdom.com]
See that? The site as-is is under 200kB per page.
What's the point of shaving off relatively nothing in the first place?
Destroying Semiconductors With Style Since 2008, and scaring you ill-educated fools since 2013.
(Score: 1) by VanessaE on Thursday March 13 2014, @05:54AM
Simple. If you only manage to shave off 1kB per page, you're already looking at 5GB/mo less bandwidth for the 5M views/month this site gets now. If Soylent can manage to get to Slashdot-levels of traffic (15M views/mo) and the savings is more substantial, say 25 kB per page, then you're talking about 375 GB/mo in saved bandwidth.
Consider for a moment how much time it would take to send 375 GB, and what else one could do with that bandwidth if it's just being wasted now.
Multiply that again by however much traffic all of the other Slashcode-using sites are getting, combined.
Also consider that not every geek has a particularly fat pipe. A few seconds saved here and there loading pages adds up.
(Score: 1) by Titanium on Thursday March 13 2014, @09:38AM
(375 gigabytes) / (10 (gigabits per second)) =
5 minutes
I fail to see your point, and I did not even multiply by the number of nodes serving up the page.
(Score: 2) by gringer on Thursday March 13 2014, @06:02AM
When I do a copy-paste of the current home page into a text editor, then count characters, I get something near 20kB. In other words, if all you had was text, your download would be a tenth of the size. I wouldn't call a 90% reduction "relatively nothing".
Ask me about Sequencing DNA in front of Linus Torvalds [youtube.com]
(Score: 2) by evilviper on Thursday March 13 2014, @06:21AM
It's "nothing" on a high-speed connection. On a dial-up connection, that could take more than 30 seconds to download. Actually, compression and caching would improve that a bit, but still...
And god help you on a 2G connection, going HALF that speed AT BEST.
Hydrogen cyanide is a delicious and necessary part of the human diet.
(Score: 3, Insightful) by hankwang on Thursday March 13 2014, @06:40AM
What's the point you ask? During my daily train commute, I pass through an area with patchy 2G coverage, with just a few kB/s transfer rate. At work (it's on a "high tech campus", well, well) it depends on which side of the building I am in.
And the layout is more suitable for mobile rendering. Although for SN the layout is manageable on modern mobile devices, you should remember that it was really written for slashdot, with SN support as a new feature.
Avantslash: SoylentNews for mobile [avantslash.org]
(Score: 1) by egcagrac0 on Thursday March 13 2014, @01:26PM
About 2 minutes per page load when the display says "AT&T E" instead of "AT&T 4G", that's the point.
Also, some of us like to save that quarter second even when it says 4G.
(Score: 3, Informative) by juggs on Thursday March 13 2014, @05:04AM
Just as a point of discussion, slashcode already has bandwidth saving options baked in.
**N.B. Before following the below suggestion bookmark this link: http://soylentnews.org/my/homepage [soylentnews.org] you'll need it to get back out of the matrix :)
Browse on over to your Homepage http://soylentnews.org/my/homepage [soylentnews.org] and check "Simple Design", "Low Bandwidth" and "No Icons". Et voila, a lot less cluttered and (opinion) cleaner look. Although you do lose some functionality as a trade off.
Maybe that could be used as the basis for a "mobile.soylent" incarnation if the need is there?
(Score: 1) by RamiK on Thursday March 13 2014, @08:19AM
I actually prefer this layout to the original and it's variations.
Hopefully it will inspire some work on the original code at least for the mobile side.
compiling...
(Score: 3, Interesting) by mtrycz on Thursday March 13 2014, @09:07AM
This comment page is TWENTY times heavier than the sourced from the digester. Most of it is cached in browser, but still on first access or on expire it will load 100kB of javascript files, that are actually pretty useless.
Maybe we can shave off from that?
In capitalist America, ads view YOU!
(Score: 2) by NCommander on Thursday March 13 2014, @03:46PM
There's some damn low-hanging fruit w.r.t. to Javascript and stuff on the site. For non-admins, I don't think we're actually using JS anywhere in the layout. backSlash goes snicker-snap if you disable JS, but I think we can safely make all those JS files conditional loads on is_admin and NOT break the site.
It WAS used for firehose, but that feature is so nackered, I don't think its coming back without a rewrite.
Still always moving
(Score: 1) by iWantToKeepAnon on Thursday March 13 2014, @04:14PM
How about adding a user-agent dropdown as part of the homepage preferences? Then for my desktop mozilla I can have the full blown experience. For my mobile browser I could pick Simple+Low/B+No Icons. When I visit preferences there could be a "default" for all UAs and a (+) button to add my current UA.
That should make a friendly "mobile" soylent site with (hopefully) little extra coding.
"Happy families are all alike; every unhappy family is unhappy in its own way." -- Anna Karenina by Leo Tolstoy
(Score: 1) by iWantToKeepAnon on Thursday March 13 2014, @07:49PM
So what is "pda", where is it set (not on the homepage prefs near No Icons, Simple Design, and Low Bandwidth), and was that an attempt at a mobile setting?
"Happy families are all alike; every unhappy family is unhappy in its own way." -- Anna Karenina by Leo Tolstoy