SoylentNews is people
SoylentNews
Such a script would ease the burden of implementing a few site-wide changes to The Global Computer Index.
As it stands 319 HTML files need to be revised in one or more of four separate ways.
Simply to contemplate such laborious and tedious work gets down so I focus on the smaller countries first, as well as the countries of whose cities I list only a very few.
I use find to produce a list of all the files that require revision. What I'd like is a script that sorts that into countries - or into US states - that have the fewest cities that require revision.
That won't save me any effort but it will make me far more productive. It's much easier for me to initiate a task if it at least appears to be a small task.
Here's some sample data:
$ find . -name index.html -exec grep -l 'Computer Job' {} \; | grep -v united | tail
./pakistan/rawalpindi/index.html
./philippines/manila/index.html
./poland/gdansk/index.html
./poland/warsaw/index.html
./russia/moscow/index.html
./russia/novosibirsk/novosibirsk/index.html
./russia/tomsk/index.html
./russia/tomsk-oblast/index.html
./serbia/belgrade/index.html
./singapore/index.html
In this list I would start with Singapore then go on to Serbia and the Philippines.
If I only needed to change "Computer Job" to "Computer Industry Job" I would use sed. But sed alone won't do it because I often have to break long lines into smaller chunks so as to make iFone Fanbois happy.
I'm also migrating my entire site to HTML 5 - but many of my as-yet-unrevised pages are _already_ HTML 5 but some get warnings when I validate them.
Some have spelling errors. Some have errors that doubtlessly would lead foreign patriots to undertake a vendetta against me, my male children and all their male children.
So really I do need to at least inspect all 319 candidate files.
I thank you, and your future managers thank you.
(Score: 2) by The Mighty Buzzard on Wednesday May 16 2018, @11:41AM (7 children)
A) That's a pointless script that won't help your productivity at all.
B) You're asking for free help on a commercial venture.
C) "Thanks a bunch" is not sufficient compensation for subcontracting something I do for a living, even if it would only take a few minutes.
If it were a script for your desktop, a service I used, or a worthwhile nonprofit thing, my answer might be different but you're asking me to perform my trade to your financial benefit but not my own. That's not cool, so no.
My rights don't end where your fear begins.
(Score: 2) by MichaelDavidCrawford on Wednesday May 16 2018, @12:58PM (4 children)
Quoting from the footer of every last page of Soggy Jobs [soggy.jobs]:
"There is _no_ charge to list your company here, nor will there _ever_ be."
If put three years of work into the site but I have yet to earn so much as a dime from it.
While I've got your attention, what leads you to believe that your script wouldn't make me more productive? I made my reason plain.
Yes I Have No Bananas. [gofundme.com]
(Score: 2) by The Mighty Buzzard on Wednesday May 16 2018, @01:33PM (3 children)
Regardless of what order you do them in, the total number is the same. Just get started instead of wasting time organizing to no end. It's a common code monkey flaw but not an excusable one in anyone out of their 20s.
I withdraw the financial objections being as they're inapplicable.
My rights don't end where your fear begins.
(Score: 2) by MichaelDavidCrawford on Wednesday May 16 2018, @01:37PM (2 children)
I implied that fact when I pointed out that such a script wouldn't save me any work.
What my requested script _will_ do is increase my productivity - and substantially so.
This because I would be far less inclined to put off small tasks than I am to put off large tasks.
How often do you _really_ write code for eight hours in a single day?
Whenever I start a new program I do so by implementing "Hello World". I then gradually - just a little bit at a time - edit Hello.c until I produce a deliverable.
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Wednesday May 16 2018, @04:42PM (1 child)
The idea of breaking up an overwhelmingly large task (update all pages globally) into smaller tasks (update all pages in $COUNTRY) is a reasonable one, but sorting by which countries need the fewest pages updated seems superfluous.
Knowing which ones need the fewest changes might let you get to those without procrastination, but the ones with more changes? You've still gotta do them eventually, and by piling them all up at the end, it looks to me like you're just guaranteeing yourself a pile of procrastination at that point.
But unlike TMB, I don't do this for a living; I'm a machinist, so this was a pleasant diversion to bang out on lunch break -- even though I'm convinced it's completely useless.
#!/usr/bin/awk -f
BEGIN {
FS="/"
}
country != $2 {
print count, country
country=$2
count=0
}
{
count++
}
END {
print count, country
}
Feed it your list of files and pipe the result through sort -n or whatever.
It's dirty -- e.g. depends on input being sorted to the extent that all ./foo/* are adjacent, so if you're concatenating multiple sets of input, pipe them through sort first. You get the idea, it'll work if you make it work. Also, it outputs a bogus blank line at the beginning. If that bugs you, I'm sure you can fix it.
(Score: 2) by MichaelDavidCrawford on Wednesday May 16 2018, @10:07PM
I'm forced to confess that the last time I used ask was in 1988.
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Wednesday May 16 2018, @05:24PM
But, but TMB, just think of all the exposure you'll get for doing it! In all seriousness though, if there is any criticism on how someone does this, then this is going straight to https://www.reddit.com/r/ChoosingBeggars/ [reddit.com]
(Score: 2) by cafebabe on Thursday May 31 2018, @03:49PM
I provided a working script with import for the manually created data [soylentnews.org] and didn't even get an acknowledgement. It may be the case that my response was missed. If it was specifically ignored and there is specific effort to re-implement this work in another language [soylentnews.org] then I'm very offended.
1702845791×2
(Score: 2, Interesting) by Anonymous Coward on Thursday May 17 2018, @01:32AM (1 child)
I second what TMB wrote above about the task.
However, what I can offer free is that if you're going to parse HTML, you need a proper HTML parser. The easy way is to use one from CPAN and Perl5. You have perl5 already there on your OS X machine. Just choose a parser and add it. The examples in the manual pages for the parsing modules will do most of the work for you.
Run everything through HTML Tidy first.
(Score: 2) by MichaelDavidCrawford on Thursday May 17 2018, @06:03PM
Most of it is valid html. I validate each page when I'm done working on it. That. Catches the occasional typo
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Thursday May 17 2018, @11:01AM (9 children)
It will be more work up front, but save you immensely down the road
You should expend your energy not on updating the template data of 300+ html files but instead you should be looking to separate the html templating from the actual data content.
Your data content (name of business, contact, description, etc) should be stored in some form of structured computer readable format (json, yaml, csv, xml, does not matter which, just that it is in a structured computer readable format).
Your templates that create the html pages should exist once, in combination with a piece of code that acts as a static site generator.
Then, when you need to change the template design, you have one set of template files to edit, and rerun the generator on the data content files.
As well, when you have to change multiple of the data files, because of the structured format, you'll nearly always be able to create a piece of custom code to change them all in-mass.
And in the end, no manual edits of 300+ files to change some spelling errors in the templates.
(Score: 2) by MichaelDavidCrawford on Thursday May 17 2018, @06:07PM (8 children)
The payload is all in table elements always with four columns
I've planned to automate it this whole time but am unclear of what I want my automation to do
Yes I Have No Bananas. [gofundme.com]
(Score: 1, Interesting) by Anonymous Coward on Thursday May 17 2018, @09:08PM (6 children)
look at static generation https://www.staticgen.com [staticgen.com]
(Score: 2) by MichaelDavidCrawford on Friday May 18 2018, @01:44AM (5 children)
(I will look at staticgen right after I post this.)
I'm going to use Subversion on a remote server so as to implement reliable storage.
I'll use Python to actually operate on the date. The first thing I want to do is write a simple Python app that will enable me to enter a given company just once, then add all their cities, states or provinces and maybe counties all at the same time, then distribute updates to all the write HTML files.
That will need to create new HTML files from time to time - Oracle is all over Creation, so presently I add just one country at a time.
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Friday May 18 2018, @01:16PM (4 children)
You do realize, don't you, that Subversion provides change history, but not reliable storage. Reliable storage is something like a RAID array to reduce the risk of loss from a disk failure followed by a backup strategy to cover for the small remaining risk of a multi-drive failure.
(Score: 2) by MichaelDavidCrawford on Friday May 18 2018, @07:36PM (1 child)
I have a raid 5 on one of my boxen
Yes I Have No Bananas. [gofundme.com]
(Score: 2) by cafebabe on Monday May 21 2018, @07:44PM
You are strongly advised to not use RAID5. On modern systems, the number of sectors multiplied by retreival fidelity guarantees data corruption during re-build.
1702845791×2
(Score: 2) by MichaelDavidCrawford on Friday May 18 2018, @11:16PM (1 child)
I don't use it often enough. Every time I want to write some Python code I have to re-read the tutorial yet another time.
I'm going to get the book Learning Python when I get paid - $$$. I gave a copy to a friend in Africa; I'm going to teach Python to him but I won't get very far if I keep forgetting all about it.
I expect I'll do enough coding for Soggy Jobs that Python finally sticks for me.
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Tuesday May 22 2018, @06:09PM
for web in python have a look at zope
(Score: 2) by cafebabe on Thursday May 24 2018, @07:49PM
It would be much easier to update your website from one CSV file. Example implementation requires make, bash and perl which are typically present on Linux and MacOS:-
(Usual instructions for uudecode process [soylentnews.org].)
Usage:-
CSV file is flat and de-normalized with following format:-
Export script contains minimal code for styling web site. You may want to improve it. For example, by adding hyperlinks to improve web site navigation.
1702845791×2