The Open Source Survey asked a broad array of questions. One that caught my eye was about problems people encounter when working with, or contributing to, open source projects. An incredible 93 percent of people reported being frustrated with “incomplete or confusing documentation”.
That’s hardly a surprise. There are a lot of projects on Github with the sparsest of descriptions, and scant instruction on how to use them. If you aren’t clever enough to figure it out for yourself, tough.
[...] According to the Github Open Source Survey, 60 percent of contributors rarely or never contribute to documentation. And that’s fine.
Documenting software is extremely difficult. People go to university to learn to become technical writers, spending thousands of dollars, and several years of their life. It’s not really reasonable to expect every developer to know how to do it, and do it well.
-- submitted from IRC
(Score: 4, Funny) by pkrasimirov on Monday June 05 2017, @10:25AM (30 children)
Good code is self-explanatory, how'bow dah?
/ducks
(Score: 3, Touché) by turgid on Monday June 05 2017, @10:27AM
Quack.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 2) by The Mighty Buzzard on Monday June 05 2017, @10:58AM (19 children)
Surprisingly, this is largely true of Rust code. It's by far the easiest language I've coded in to deal with undocumented or poorly documented code.
My rights don't end where your fear begins.
(Score: 1, Insightful) by Anonymous Coward on Monday June 05 2017, @01:25PM (1 child)
Wait until the dregs show up; they always do.
(Score: 0) by Anonymous Coward on Monday June 05 2017, @03:45PM
yeah, cuz you're a zealot if you're not a sycophantic whore.
(Score: 4, Insightful) by DannyB on Monday June 05 2017, @03:23PM (16 children)
Bad code can be written in any language. When you've been around long enough, you eventually see everything. Nothing surprises you any more.
IMO:
0. Change your mindset. You're not writing code for the compiler. You're audience is another human being -- not a compiler. Once you get this philosophy your code improves a great deal. As you write every function you're thinking about the person reading your code.
1. invest time into using good identifiers. Class names. Method names. Parameter names. Local variable names to a lesser extent. Good identifier names become a significant amount of documentation.
2. well documented code involves trying to make explanatory comments unnecessary. Comments explain things that the reader cannot understand from the identifier names.
3. Avoid overly clever terse one liners. These conceal a lot of what is going on. I would rather have a couple or three lines that make perfectly clear what is happening, where the compiler can make the obvious substitutions that don't need to happen in the source.
4. DO write documentation in your language's documentation system. Eg, for Java use JavaDoc. Or similar mechanism in C#. Specially formed comments on classes, methods, etc are automatically gathered and generate hyper-linked documentation that is extremely useful. Once you've used other people's good "javadoc" in your web browser, you are encouraged to write your own useful comments. But only say what is necessary to say. The identifier names should do a lot of the communicating.
5. Write test cases in a way that serve as examples of how to use specific functions, classes, services, or whatever you are testing. These tests also become (gasp!) a form of documentation.
There's all the other obvious stuff. Make proper use of things like information hiding, immutability, functional style, object orientation, inheritance, polymorphism, etc. None are exclusive to the others. All of them have a proper use.
Use the right language for the right job. If there were one perfect language, everyone would be using it already.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 0) by Anonymous Coward on Monday June 05 2017, @04:26PM (13 children)
I take small issue with this, small comments save a lot of time. Instead of reading through all the code to see what it does you can first read the comment that tells you, then focus on understanding how the code works. Searching back and forward for how a variable is used can be a pain, but a small comment can explain the most important details.
(Score: 2) by DannyB on Monday June 05 2017, @05:16PM (12 children)
If it is not obvious, then the comment would be necessary. Nevertheless, I would strive to make the comment unnecessary if possible.
Another thing is that experience comes as much from reading other people's code than writing code. You gain the insights in what you struggled to understand.
A good IDE can trivially highlight all of the uses of a variable.
Saying that you prefer a simple text editor rather than the distracting noisy blinking lights of a modern IDE is like saying you prefer to dig ditches with a shovel rather than the distraction and noise of a backhoe.
Both the IDE and backhoe have a steep learning curve. The simple text editor or shovel require no learning.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by tangomargarine on Monday June 05 2017, @06:12PM (11 children)
Would you consider emacs and vim simple text editors?
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 2) by Zyx Abacab on Monday June 05 2017, @06:49PM (4 children)
Then again, neither vim nor emacs is only a text editor. One is a programmable command environment, and the other is almost a full-fledged OS; in either one, the text-editing functionality is merely a coincidence.
(Score: 2) by tangomargarine on Monday June 05 2017, @06:55PM (3 children)
That requires a certain perspective to get there. On a descriptive level, that may be what they are now, but the fundamental purpose was always to be a text editor. And I would argue the raison d'être can't be a coincidence, by definition.
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 2) by FatPhil on Tuesday June 06 2017, @07:28AM (2 children)
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2) by tangomargarine on Tuesday June 06 2017, @02:42PM (1 child)
If by "legacy feature" you mean "something I still use on a daily basis in a non-ironic fashion," I guess.
- a 27yo emacs user
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 2) by FatPhil on Wednesday June 07 2017, @08:21AM
(I had uEmacs on my Atari ST a year or two before that.)
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Insightful) by DannyB on Monday June 05 2017, @09:06PM (5 children)
On their own, no. Definitely not simple text editors.
Compared to IDEs like Visual Studio, Eclipse, IntelliJ, Netbeans, yes I would call them simple text editors.
I'll focus on Eclipse since that is what I use. But the others I mention will have similar capabilities.
The IDE has deep integration with the compiler. The compiler is actually a library with an API. The IDE silently builds a database of everything about code. If I click on a variable and click a button to show me all uses of that variable, I'm not just seeing the results of a dumb text search. I'm seeing what the compiler understands to be a reference to that variable. Not a local variable with the same name. Not something with the same name in a comment.
Many different classes may declare a method called foobar() with the same type signature. If I see some code like . . .
frob.foobar()
and I click on foobar, and hit F3 to take me to the declaration of foobar(), the compiler (and thus editor) are not confused about exactly which source code declaration to take me to. The variable frob is of some type, and from that type the IDE can deduce where the foobar() method is declared. Exactly and precisely. If I click in foobar() and hit Ctrl-Alt-R (to rename) I can rename foobar(), but as I type, character by character, all references anywhere in my very large project are all simultaneously renamed. And again, exactly and precisely. Other "foobars" that are not the exact one I am renaming are unaffected. This is not a search / replace type rename, it is done with deep knowledge of the source code as understood by the compiler.
Suppose I'm looking at a function with over 200 lines of code in it. Too many. I see a group of eight lines that could be factored out into an independent function, I select those six lines, right-click, pick a refactoring operation to extract those into a new function -- and like magic -- a new function is written right below the function I was looking at. Those eight lines are in it. The function has all the right parameters. The original eight lines are replaced by a call to the newly written function. And of course, if Eclipse couldn't actually do this refactoring operation, it would have said to.
There are many other powerful tools in a modern IDE that tremendously help you write your code. Automatic generation of countless templates and idioms that you type and re-type.
Hit a button to create a for loop. The template appears right in the code where you were:
for( int i = ___; i ____ ; i++ ) {
}
the "i" is hilighted, and already in "rename" mode, so as I type, the "i" changes into whatever index variable I want. As I rename it, all three "i"s in that statement are renamed together. (And any others in the project would be also, except the visibility of that variable is only within the body of the for loop.)
Now I don't pretend to know the full capabilities of vim or emacs. I use vim from time to time. I understand emacs has a custom lisp, and every keystroke is really just triggering a lisp function bound to that keystroke. So I understand they are powerful in a way. But I don't believe they do the kinds of powerful operations I describe above -- and way, way more.
In Eclipse, for example, I could be working on code for Java, Clojure, Python, C, Kotlin, and on and on, other languages.
The template for the for loop above is pre-defined. But it isn't special. You can define your own code templates, and indicate variables, types and other information so that when the template is inserted it is ready for you to fill in the blanks, and it will properly rename variables within your template.
Documentation for everything is rapidly available. I can get the JavaDoc viewer for any function or class that has one. I could go on.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 3, Interesting) by coolgopher on Tuesday June 06 2017, @06:05AM (4 children)
A nice IDE can be, well, nice. In fact, I'd say that they're commonly *too* nice. They molly-coddle you and make it trivial to jump through 16 levels of function calls across a dozen source files (or even components). And this is a problem in my opinion. A developer is not penalised for writing convoluted code. And in my opinion, in the vast majority of cases, they should be. Make it a pain to follow the call chain, and people will have an actual incentive to write simpler, clearer code. As you said earlier, the target audience should be human, not machine.
I spent a number of years in IDEs, but I have gone back to vim & make where possible. You easily get lazy when in an IDE, and your bigger picture understanding of the project starts to deteriorate. I *like* having to have a mental model of the whole thing, it makes it possible for me to make good calls about things like object ownership strategies, threading and locking, and other deep aspects. With an IDE it's so easy to instantly find yourself in exactly the right place and never have to pay any attention to the surrounding code/environment, and that's bad for your understanding.
You mention the template-type macros. I personally really dislike those. They disrupt my flow of thought. As I'm opening a loop, it's almost a meditative exercise during which I have time to sharpen my intent of *exactly* how this loop is going to behave. The whole thing flows both visually and mentally, whereas when I've tried using template-macros, it jars and skips both visually and mentally. Presumably it's something that one would get used to, but I'm not feeling there is much benefit to be had in that particular area. My typing speed is never the limiting factor when coding.
And then there is my REAL bug-bear with every IDE I've ever used. They're a total !&#!% nightmare to automate! The power you might have available within a keystroke or two while in the IDE, is locked away completely from the outside. Build flags are commonly jammed into xml madness deep within some IDE directory structure, and programatically adjust one while the IDE is running is typically impossible. Even choosing the build profile can be difficult or even impossible from the outside (have these people never heard of CI or even nightly builds?!).
IDEs are wonderful when you're first starting out, but in the end they alway start to feel like a prison to me.
(Score: 2) by urza9814 on Tuesday June 06 2017, @02:23PM (1 child)
THANK YOU
I find my preferred editor is Brackets. Used to use Kate, still like it a bit better (NOTHING handles opening multiple projects/documents as well as Kate did), but I don't use KDE anymore and Kate was getting a bit buggy for a while anyway. Both are text editors with a few IDE features -- syntax highlighting, tabs, stuff like that. Unfortunately even they generally try to be too goddamn clever these days. Who the hell thought it was a good idea to code with friggin' AUTOCORRECT and why do even the most basic text editors enable that by default now? They don't do it for text, but they do it for code...what the hell? I'm sick of typing a loop only to have the script fail because my editor decided it needed to throw in some extra loop closures for me. Who are these systems designed for? What coder is such an atrocious typist that typing a damn closing parenthesis is something that needs to be automated? All that does for me is break my concentration and waste effort...I've gotta type it, then I've gotta notice something went weird and reanalyze the line I just typed to get everything matched up again. I cannot conceive of any reason why anyone could possibly believe that was a good feature to implement, let alone making it default behavior...
Of course, I do mostly scripting languages lately, so there aren't many full IDEs available and the more advanced features aren't needed. I occasionally do some Java but nothing big...I wouldn't consider writing a LARGE (ie, more than just me) Java project without using Eclipse...but I also wouldn't use Eclipse without disabling a bunch of garbage that just gets in the way. A learning curve is one thing; unpredictably doing shit on its own that the operator did not intend is something else entirely.
I like "The Unix Philosophy". I like small tools that do one thing and do it well. I don't want a software development walled-garden "ecosystem", I want *tools*. It's like at work we use this software called Informatica, it's kind of a drag-and-drop IDE for large database ETL operations. And it's the worst thing I've ever worked with, because if their tools don't have a feature you need, you're just screwed. It outputs code as XML files, and if you export the exact same code twice you'll end up with two different files. It'll output stuff in different orders, give components different names, add or remove extra null attributes on tags...if you want to compare changes between two versions the only option is load them both in the IDE and manually compare one section at a time. Their IDE doesn't have any kind of diff feature and the exported XML is too variable to use standard tools like diff. And even the features it does have don't work -- you'll open a transformation, copy some text from it, close it and open the find box and paste that same text right in there...and it won't find it. If you have specific text to find you can kinda do that in notepad on the XML, then you've gotta trace back the hierarchy, which will give you something that you can put into the Informatica find function that it *might* be able to work with, then you can go back there to try to look at the actual code...it's a goddamn mess. Not to mention that most configuration values can be changed or overridden multiple places, without any indication of which one is actually going to be used...but that's another topic probably.
Highly integrated environments also tend to be highly restrictive. They only help if you want to work exactly the way they expect you to work. Sometimes that can be an advantage I suppose...for large projects you want consistency in coding styles and tools so if you build the project such that it forces a particular IDE, and that IDE forces particular tools and styles, that solves that issue. Like my problems with Informatica...I'm pissed that I can't use diff, but I'm probably the only person here who would even bother to try. Everyone else just goes "Can't find what changed, let's schedule a meeting." But for a *personal project*, I want to work the way I want to work, fuck the IDE that's trying to get in my way.
(Score: 2) by tangomargarine on Tuesday June 06 2017, @02:55PM
It's kind of weird -- auto-parentheses seem natural to me in Eclipse, but when I enabled a mode in emacs to do it, it drove me nuts for a couple days until I turned it off again. Maybe it's a C++ vs Java thing, or a workflow difference...
It's too bad code tags in emacs (the equivalent of F3 in Eclipse) look like such a mess. This being a Linux tool, of course there are 3 or 4 different ways to do it.
Wow. Sorry :-/
Kind of like how the search in Windows is so dysfunctional sometimes I can search for the name of a file in the same directory and it won't find it. So rather than even try, I just fire up Cygwin these days.
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 2) by DannyB on Tuesday June 06 2017, @04:26PM
A bad developer can (and will) write bad code using any tool, any editor, any programming language. IDEs are completely irrelevant to that complaint.
Being able to jump through and analyze complex (and sometimes horrifical) call chains is a good thing if you're trying to understand someone else's bad code.
Don't blame the tools. It's like blaming the tools where you should blame a bad mechanic.
I absolutely agree bad developers who build convoluted code should be penalized, but not by the IDE or lack of one. What you have here may be less of a technical problem and more of an HR or management problem.
I like that too. Fortunately, for me, that is possible. It may not be possible for all people. Again, the IDE seems irrelevant. Or if it is relevant, I would argue that the IDE makes it easier to have a mental model of the whole thing, because the whole codebase is much easier to analyze.
What if you had a very large project written by many people, and the whole thing was too complex to fully understand? Would you still not use a modern IDE? Or would you look for a job with a simpler code base to work on?
To each their own.
Maybe you're doing it wrong. I love automation. I automate anything and everything possible.
The IDE certainly doesn't stand in the way of my build process which ultimately ends up producing a single file. (A WAR file or JAR file depending on which build target I choose.) I have a nice deploy process where the IDE is irrelevant.
I found exactly the opposite. I was intimidated by IDEs, and so I started with simpler tools. Then I used the IDE, but as just barely more than an editor. Then as I learned more and more about how to use the IDE, I came to appreciate its power.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by termigator on Tuesday June 06 2017, @07:44PM
I generally agree with your comments. Vim is still my main editor and still make use of grep when searching for things. I am way more proficient editing in Vi/Vim then any other editor.
I have worked with IDEs like Eclipse and Netbeans. They have their uses, but when all the "magic" they do goes wrong, it can go wrong badly. This can cause one to fight the tool to get productive again, losing time getting actual project work done. I will load up an IDE to help with things like code name refactoring (a rare need) or for real-time debugging, but day-to-day, I do not use any and I make sure the projects I am lead on do not depend on any IDE.
A problem that IDE dependency causes is developers become to dependent on it, where they can no longer be productive without it. One time, I had to set up Eclipse for a project (the project was not dependent on any IDE) for a new hire since that is what they were used to. The person did not even know how to set up Eclipse for themself. IDE heavy dev environments lead to many folks being ignorant of how things actually work. If those folks move to a different environment with different tools, they are completely lost.
Any one who programs should at least know how to compile and run a hello-world like program in the language of choice the project uses.
Any project should never be dependent on an IDE. Project's should allow developers to choose what IDE they want to use (or not use). In the project I am lead on now, I have two other developers using Eclipse while I do things old-school.
Side comment: For Vim users that have to work with Eclipse, there is eclim, IIRC, that runs Eclipse in server mode, allowing you to then have Eclipse services within Vim interface.
(Score: 2) by goodie on Monday June 05 2017, @04:46PM (1 child)
While I agree with this for an API, there is an issue with regards to JavaDoc in my opinion. Many times, there is too little documentation on the overall context of the application, just a list of hyperlinked properties and methods that say little on the "overall" way the system works. This is where your next point comes in handy (examples). In my opinion, this is where a place like MSDN does pretty well. You have the API doc, some contextual information, and at the bottom of the page, you get examples. To me, this is good documentation. But of course, it takes time to create and maintain.
Fuck yes! And keep it up to date. As a developer, there is nothing better for me than to read an example because it also helps me figure out *how* it should be done (if it's well written of course).
The thing is that writing documentation is only really working when you do it as you go, not as an afterthought once you are done. By the time you are done, your system is already doing what you need and you've moved on to something else (the thing for which you created the system to begin with!). I've seen many oss projects where I've had to go on some random person's blog to try and understand how it's supposed to be used. I don't like it, for all I know the guy is giving garbage advice and I wouldn't be able to know it.
(Score: 2) by DannyB on Monday June 05 2017, @05:23PM
I have sometimes written simple examples, and fragments, in the JavaDoc documentation. Thinking of the poor sop who may one day maintain this code. And it will be around a long time. I will probably expire before it does. Code I wrote in the early 80's wasn't phased out until the early 2000's -- also forcing us to deal with Y2K. Code I wrote in a newer language in the 90's is still being used today, but being phased out by encouraging upgrades to what I work on today.
If they are not up to date enough, then they fail. You have to fix them, and in the process you are fixing your examples of how to use the system being tested. You can also write JavaDoc comments in the tests. :-)
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by JoeMerchant on Monday June 05 2017, @01:09PM (8 children)
That, and do they really not know how to write docs, or just not care?
Coding is challenging, people dive into it with something to prove: they are masters of that universe.
Some people will take up testing as an obsession, good testing is far from trivial and very valuable - especially to keep the coders from screwing up the existing value when they extend it.
Documentation - lacks challenge, except perhaps in language translation, and learning more than a couple of human languages is harder than most people want to take on.
🌻🌻 [google.com]
(Score: 2, Insightful) by Anonymous Coward on Monday June 05 2017, @01:40PM (2 children)
Unfortunately, too many people believe that.
It's not just expressing something in a human language. It's expressing it comprehensibly, efficiently and unambiguously, in a way that lends itself to translation.
Turns out, this is basically a hard, AI-complete problem. Most devs don't have those skills. In fact, it's very hard to hire for deep technical writing positions because you need someone with technical and writing chops - and that's a very rare combination.
The real problem is that doing a good job of this sort of writing can be a drag, and devs want to do what they find to be fun, not a drag, and there's nobody to hold their feet to the fire.
(Score: 2) by Grishnakh on Monday June 05 2017, @02:42PM (1 child)
It's not just Free software that suffers here though; proprietary software is no better for the most part. Quick, pull out your Windows 10 user's manual... oh wait, such a thing doesn't exist.
The problem here, as I see it, is that technical writing is not rewarded or valued appropriately, so no one really wants to do it. The people who write software get paid well and have an interesting job (because they're creating something new), so a fair number of people flock to that profession (or do FOSS projects on the side). The people who test software generally get paid well too, so some small number of people go into that, and that's a really interesting profession too. But writing about someone else's work isn't very glamorous or fun, but worse how well does it pay? Terribly. So who's going to get a technical degree, plus a language degree(s), to be highly qualified for this job?
(Score: 0) by Anonymous Coward on Monday June 05 2017, @10:39PM
How have so many people been indoctrinated to the point where they believe you need a degree to do anything? It's not as if knowledge only exists in colleges or universities. Lazy know-nothing employers create this situation by mindlessly requiring degrees for jobs that simply do not need them, and then they end up hiring scores of idiots who do have degrees because most schools have next to no standards anyway.
(Score: 2) by DannyB on Monday June 05 2017, @03:28PM (4 children)
I wrote several items I believe in a different comment. But one worth repeating here. Change your philosophy of programming. You're not writing code for the compiler. The compiler is not your audience. Another human reader is. The exercise of programming is not merely to get successful execution on a machine. Programming is about explaining to another person how this works -- the fact that a machine can execute it is secondary.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 0) by Anonymous Coward on Monday June 05 2017, @04:27PM (1 child)
I know I‘m being pedantic, but I disagree with the last comment that working code is secondary. Executimg code is primary. Although I totally agree with you otherwise. And I don't think this makes writing understandable code secondary either. I would argue that part of executing code is fixing/maintaining that code which includes making it easy to understand.
(Score: 2) by DannyB on Monday June 05 2017, @05:25PM
You misunderstand to say working code is secondary. Working is the number one goal. If it doesn't work. Then there is no point. But the compiler processing your code is secondary to a human being able to comprehend it. The compiler doesn't count. Of course it has to work. But if it works and nobody can maintain it, then it also fails on an important level.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by Azuma Hazuki on Monday June 05 2017, @05:56PM (1 child)
Strictly speaking, the compiler *is* your audience, but so is anyone else who has to maintain it. In effect, you're trying to express yourself in a way the compiler can read that also makes sense to a human. We can trust the compiler to break the code down to its component machine-code no matter how it's written, so long as it's written correctly; for the sake of other humans, though, you should pretty-print it.
I am "that girl" your mother warned you about...
(Score: 2) by JoeMerchant on Monday June 05 2017, @07:26PM
Documentation isn't just for programmers, either. A vast swath of open source code completely lacks user facing documentation.
🌻🌻 [google.com]
(Score: 2, Informative) by Anonymous Coward on Monday June 05 2017, @10:29AM (9 children)
It's not even that they don't know. It's just almost nobody wants to.
(Score: 4, Interesting) by The Mighty Buzzard on Monday June 05 2017, @10:55AM (3 children)
Yup. It's pulling teeth to get anyone on staff except audioguy to even document our own setup. Yes, this includes me.
My rights don't end where your fear begins.
(Score: 2) by JoeMerchant on Monday June 05 2017, @01:13PM
Documentation is a thankless job - you do a great job of documenting the existing setup, then the coders and testers change it all and your documentation is broken, but nobody cares enough to help you fix it.
Autogenerated documentation like doxygen at least moves the problem closer to the source... still impossible to get coders to slow down and update the comments even when they're right there in the header of the subroutine, but when it's broken, the git-blame function at least gives you a name to go after to try to get some "what the hell were you thinking here" to go with the obvious stuff.
🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Monday June 05 2017, @01:39PM
It's the least fun part of your job. I barely do it when they pay me, why do it for free?
(Score: 0) by Anonymous Coward on Monday June 05 2017, @02:38PM
I would love to write documentation if someone payed me to do it.
but I'm actually payed for running the code I write and analyzing the results.
(Score: 2) by driverless on Monday June 05 2017, @11:31AM (4 children)
Yup. I went to a talk about a decade ago that compared OSS to commercial software development, and one of the points they made was that in a commercial environment you get (mostly) decent documentation because you can pay someone to do it. In addition because you've got paid features in your product you want your customers to know they're there and how to use them, so they need to be fully documented.
Without that financial incentive, documentation is treated as job #37. Or as they put it, "everyone wants to be a code god, no-one wants to be a documentation god".
(Score: 2) by aristarchus on Monday June 05 2017, @05:08PM (3 children)
Except that with commercial (proprietary) software, no one can see the documentation. So we really never will know how complete or competent it is. Windows is a case in point.
(Score: 0) by Anonymous Coward on Tuesday June 06 2017, @01:36AM (1 child)
Go onto any Windows computer and press F1 at almost any time. That's their documentation. And that's what comes of following standards. Luckily must programs have yet to crap all over Window's old shortcut standards even if they did abandon everything else. Sadly many companies are turning those documents into online sites, so you can't access help when offline or when their site is being updated, but at least the shortcut key hasn't changed.
Not all documentation is JavaDocs.
(Score: 0) by Anonymous Coward on Tuesday June 06 2017, @09:23AM
For most Microsoft software it sucks. It used to be that hitting F1 would take you to the specific paragraph that explained whatever window you are currently in. Nowadays, if you're lucky, you'll get the main index.
If you are not lucky, you'll get something called "navigating help".
Visual Studio is the exception. Pressing F1 on any system class, property of method 98% of the time goes straight to the MSDN page for that class/property/method. And most of them actually do explain what it does, though some unfortunately only contains the same words that were combined to give the class/property/method name.
(Score: 2) by driverless on Tuesday June 06 2017, @03:15AM
So you can't see any of the tens of thousands of pages of MSDN, Technet, or Windows Help? Does your cult prohibit you from reading non-GPL'd docs or something? In any case though the qualifier wouldn't be "no-one can see it", it's only "members of my cult aren't allowed to look at it".
(Score: 3, Insightful) by FakeBeldin on Monday June 05 2017, @10:54AM (5 children)
Writing good documentation is like writing a good program.
It seems simple when you start, then it escalates, then you realise you maybe should've planned it out a bit before starting, then you go back to the drawing board, then you salvage what you can of what you have produced so far.
Is it any surprise that someone who went to this process already for one goal is not eager to go through it fully again?
Is it any surprise that they take the easy way out? No, sir, it isn't.
Why does this particularly affect open source software?
Because the challenge of writing a program or fixing a bug can be intellectually rewarding on its own, but writing the proper install guide that is easy to read isn't really. Moreover, buggy programs are also buggy for the creator, but buggy docs might be crystal clear to the creator.
The people who like explaining things find a job or hobby to do so face-to-face. Not by sitting at home, writing for hours and hours and receiving scant positive feedback.
(Score: 3, Interesting) by JoeMerchant on Monday June 05 2017, @01:18PM (3 children)
A proper install guide may be intellectually rewarding (to some) but it's socially thankless.
In essence, documentation is "programming the users" - which is orders of magnitude more difficult than computer code. You can "get it right" for one set of people and completely wrong for the next batch, not because what you wrote is incorrect, but because the next batch interprets it differently. Expand your verbosity to cover multiple target audiences and they all lose interest: TLDR... no win.
🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Monday June 05 2017, @03:10PM
What we need is a high level language for documentation, something that lets you write it once and then compile it for all users, instead of writing our documentation in "user level" code.
(Score: 2) by DannyB on Monday June 05 2017, @03:40PM (1 child)
I tend to think that if you need much of an install guide, you're doing it wrong.
The development team, or perhaps a separate installation team should do the heavy lifting of doing the install. How difficult is it to install Firefox? Chrome? Both of these are extremely complex pieces of software. Extremely complex.
Try to turn installation issues into configuration issues.
Use standard installer mechanisms for the OS, platform, language, environment, etc.
An example of a commercial web application I developed, that, at one time, the customer could host on the customer's own equipment. Installed by their own IT people. Installation is like this:
1. run setup.exe
2. Next, I agree, Next, Next, (wait), Finish
3. Server is now running. No reboot required. (Service can be seen in Windows Services control panel.)
4. Point web browser to server. Get "Out of service" page. Use special URL to get to initial web based control panel.
5. Use web based control panel to enter activation code, database connection information, and other very basic configuration.
6. End result, the server is up and running, presents normal login page, ready to log in and start creating other user accounts.
Now that sounds easy. But the development was not so easy. It was a hurdle to make the installation and set up process so easy.
Now the above could be edited to replace "setup.exe" with "deb install file". Replace "Windows Service control panel" with "service" command. Etc.
One further thing I learned (but have not yet done) is from a Mozilla video I saw some years back. Summary: don't build a product, build a pipeline. After the user installs it, it should be able to phone the mother ship and update itself. As easily as a TiVo does. Or Firefox. Or Chrome.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by JoeMerchant on Monday June 05 2017, @07:23PM
I agree, install guide is a bad example, software literally should install itself properly with zero intervention in all systems that even resemble a standard configuration. Maaaaybe have a few options for "advanced" users as to how invasively it integrates into handling certain file types or how many launcher shortcuts you want auto-created.
However, the principle still applies to all forms of documentation: you're giving instructions or explanations to people, not machines - and people are much more difficult to communicate a precise set of information to. One set just needs to see: "BSD standard implementation." anything more is wasting their time, another set needs every single feature spelled out step by step with examples.
🌻🌻 [google.com]
(Score: 2) by tangomargarine on Monday June 05 2017, @02:34PM
Because we can actually see the source? I'm not assuming this doesn't happen for closed-source projects, too.
"Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
(Score: 4, Informative) by ledow on Monday June 05 2017, @11:07AM (3 children)
Say no more.
I could not find any reasoning or explanation for how it's supposed to be used, only example code that was incomplete.
Literally, there is no documentation (or even function) on how to validate a signed certificate, and the possible results. You have to imagine everything that could be wrong (certificate expired, signing certificate not valid, certificate corrupt, signing certificate doesn't actually sign the other certificate, one has a lower security than required, etc.) and work out how to check and code for it.
Most libraries at least have a few helper functions. With OpenSSL it's copy the incomplete examples (and hence introduce vulnerabilities) or do it yourself.
Literally, I just wanted to generate a certificate, sign it with another, then bundle both with a program that then checks the former is signed by the latter. It ran to hundreds and hundreds of lines of code building on the examples after discovering so many ways to "fool" the examples, and having to rip them apart to understand how they worked, splice them together, add extra checks, etc. And I wasn't even doing anything complicated like trying to do this over-the-wire, just literally from a few certs saved on disk (the idea being I could keep a CA cert for myself, sign a customer's software with it, and have the software check that it was signed and valid (i.e. in-date) before continuing). Not defcon levels of security, by any stretch of the imagination, but it was far more hassle than it was worth and also I could never be sure I'd got everything right.
And there is ZERO useful documentation. Even other people's tutorials were far from useful in such regard (literally copy/paste the example code and use).
Lack of documentation certainly hurts, but it's not a magic cure-all by any means. However, it does show you quite how much design effort, user-interaction, and even end-user use the library gets. Libraries like SDL have everything documented, lots of tutorials and third-party docs available, etc. and examples that cover all kinds of scenarios. That proves that people often code against them directly.
OpenSSL has pretty much nothing. Which to me says that almost nobody codes against it directly, and everyone relies on bigger projects, bigger libraries, or wrappers around it to do what they want. To me, that indicates that such wrappers should be PART OF THE LIBRARY.
(Score: 5, Insightful) by driverless on Monday June 05 2017, @11:35AM (1 child)
What do you mean "no documentation"? Here's all the OpenSSL docs you'll ever need [peereboom.us].
(Score: 2) by ledow on Monday June 05 2017, @12:41PM
Cheers for that. If that link doesn't prove my point, nothing does.
And I can't imagine all that much has changed even with all the problems with OpenSSL discovered in recent years.
(Score: 2) by JoeMerchant on Monday June 05 2017, @01:22PM
They claim it's not security by obscurity, but in practice, it is. The flaws are hard to find, but they're there waiting for exploit.
🌻🌻 [google.com]
(Score: 3, Informative) by kaszz on Monday June 05 2017, @11:27AM (27 children)
Crappy documentation has been around for decades. Nothing new ;-)
But one thing I often miss is that a lot of (free) software writers won't even begin to tell what their program is for even with like 4 sentences.
Like "ls small program to list files in a directory with lots of options."
(Score: 3, Touché) by turgid on Monday June 05 2017, @12:27PM (5 children)
man ls
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 2) by kaszz on Monday June 05 2017, @12:56PM (4 children)
Not all programs have man pages. That's when you get into the hmm territory.
(Score: 2) by turgid on Monday June 05 2017, @02:33PM (3 children)
None of my programs have man pages. That would be another boring and stupid markup language to learn and more stuff to keep up to date :-) And if you write a man page some poor soul might get the mistaken impression that your code actually works and can be used...
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 0) by Anonymous Coward on Monday June 05 2017, @07:15PM (1 child)
FWIW, if you write documentation in almost any form, your build scripts can use pandoc to change it to man pages.
(Score: 2) by kaszz on Monday June 05 2017, @11:43PM
That is time spent learning the idiosyncrasy of those build scripts and pandoc that could been spent on writing code.
(Score: 2) by kaszz on Monday June 05 2017, @11:39PM
Make a simple text file. It solves a lot for very little effort.
(Score: 2) by VLM on Monday June 05 2017, @01:16PM (6 children)
I'm not necessarily disagreeing, but its hardly limited to FOSS.
If you are unfamiliar with vmware's products lets try a little game, you have five minutes to figure out what the vRealize product does. Not accepting an answer of manager bullshit like "improve efficiency, performance and availability" which is the most informative line of bullshit bingo cut and pasted from the entire marketing page. I'm asking what does it do, like emit a close analogy or describe a workflow or similar.
Good Luck. I'd give less than 50:50 odds a question can be answered that fast. There's a lot of marketing BS to wade thru.
I use it, and I'm not even entirely sure what the answer is, so this is non-trivial question.
I've seen this in more enterprise-ish FOSS also. Stupid as hell codewords. Lets look at openstack. Is Glance the object store? Naw dog its the DBaaS. Sorry DBaaS is Trove try again. CLOUDKITTY? seriously, thats your project name? WTF WTF WTF WTF abort abort just write a huge check to vmware and at least you'll eventually figure it out. Whats the Openstack update manager project name, let me guess "wipeToBareMetalAndInstallESXi" Stupid subproject names.
I've used both and because openstack isn't working so hard to nickle and dime you, the integration is stronger and much smoother, but the coolest marketing features of vmware don't (yet) exist in openstack. Ironically openstack is much more confusing because they're not nickel and diming you there is no way to set up ten levels of something "super lame and simple" like in vmware.
(Score: 2) by choose another one on Monday June 05 2017, @03:52PM (5 children)
> I'm not necessarily disagreeing, but its hardly limited to FOSS.
No it's not, but at least with commercial there is potentially some money in the budget to pay for docs. There is also an incentive, up to a point - better docs mean fewer calls to support means less cost more profit (assuming annual support contracts not pay-per-issue, and assuming the docs are not so good no one takes support).
With FOSS, since the product is typically free there is no money to pay someone to write docs, the programmer doesn't do it for free since there is no "itch to scratch" (they know how it works), and there is no potential revenue benefit - in fact a potential loss as people are then less likely to come to the programmer for support (or enhancements). Similarly a documentation writer is not going to do it for free, because they don't know enough unless they buy the programmer's time to explain.
Of course then there is big-commercial-FOSS like Openstack, which is somewhere in between - with an actual team and budget that may support docs, but no revenue incentive for it.
(Score: 3, Interesting) by VLM on Monday June 05 2017, @05:03PM (4 children)
That brings us back to what does better mean. For a laugh try my "vRealize challenge" and try to decode the marketing-speak in less than five minutes.
I'd say in excess of 95% of the vmware website is useless. Its even funnier in that the target is moving fast that esx 6.5 can't really be used with 5.0 docs. A practical example is the legacy FLASH web ui for vcenter is the only option for entering license keys with 6.5, you can't enter license keys using the HTML 5.0 web interface and the instructions are INFURIATING as anything older than six months on the vmware website or "helpful" blogger posts on the internet don't acknowledge the fact. But if you're running 6.5 and can in freaking 2017 find a way to run a web browser with legacy old fashioned FLASH, and then use the legacy FLASH web interface, then and only then can you enter licensing key information into vCenter. Otherwise the html 5 UI is visually identical other than that option is missing. Hilarious, huh? The entire system is like that, top to bottom across the whole field. Nothing but the equivalent of "in-jokes" as far as the eye can see.
I've had some hilarious adventures converting standard switches to distributed switches, very herculean efforts if you're playing "stupid VLAN games". From memory you can only configure or use distributed switches from vCenter/vSphere flash web page UI but you can only add and remove standard switches from the individual ESXi host web interface. So yeah there's three web interfaces for "the same thing" there's the ESX host UI, the flash legacy UI for vsphere, and the vcenter html5.0. And some tasks can only be done in 1 or 2 of the three UI. Hilarious. And the way its documented is you figure it out the hard way. Which is why VCP cert people, etc, earn such big bucks.
In fact if you're playing stupid VLAN games you can't install the second half of vCenter without some interference to the ESX host its being installed upon, it'll insist on dropping on the probably standard switch on vlan 0 regardless what vlan your mgmt network actually uses. And then there's tribal knowledge thats only semi-documented (well, maybe its in release notes now) like configuring more than one DNS server crashes the vCenter installation (WTF?)
Also sometimes vSAN needs dead chicken waved over it. vSAN why are you marking a disk as unhealthy for no discernible reason, why why why ...
vSAN is also hilarious in that AFAIK the only way to create a new vSAN is to do it as part of installing a new vCenter appliance. I know the html5.0 web UI can't create a cluster using vSAN (I think?) maybe there's some way from an already installed vcenter using the legacy FLASH web UI?
The joy of scratch partitions and log folders is legendarily hilarious. Just don't even go there.
You can guess what I've been playing with for the last two weeks in my spare time. VMware is aggravating and ungodly expensive, but fun when it works.
I have seen this in decline with the rise of unit testing and halfway attempts at unit testing. Then its a pretty short step from "cut and paste this test from unit testing into the docs file and call it a verified and tested example and we're done here" I've seen worse.
I've also seen programmers "hate writing docs" but they really hate writing corporate-ized docs. Real docs not so bad, most of the time.
I've been programming stuff since '81 and as I've become a better programmer I've gotten better about writing docs. It might be a condemnation of the entire field that when everyone's a noob, everyone writes docs like a noob, but as the field matures the professionals separate out and docs just aren't that bad. If you can't express yourself in Her Majesties English then you sure as hell can't express yourself in Clojure or SQL.
(Score: 2) by choose another one on Monday June 05 2017, @06:27PM (1 child)
Yeah, maybe if I had said something like "more useful" that would have been better... :-)
Decoded - the answer reduces to "it's marketing BS".
Actually it looks like it's a sort of management console for creating, managing and monitoring "everything" from bare metal (obviously it can't create bare metal, but probably just gives an undocumented error number if you try) through virtual stuff to cloudy shite (and hybrids of). It therefore gives you one unholy mess of a tool to do everything badly, it won't give you all the options you'd have by doing things directly using individual tools, yet it is almost certainly a cobbled together collection of individual tools itself - and they probably don't look alike, function alike or indeed have any useful commonality. On top of that it throws templates and things so you can do more stuff badly and with even less flexibility - VMWares idea of what you want rather than what you actually wanted. Then the UI(s) are probably so bad that it needs APIs/Orchestration (i.e. scripting) to do do anything complex - by the time you've learnt that you might as well have just scripted the underlying stuff directly.
How did I do? Took longer to write that description than to read marketing :-)
Me too, in fact it is more or less backed into TDD definition - the test is the spec for what the unit does and is therefore the documentation. At least the docs can't get out of date I suppose.
(Score: 2) by VLM on Monday June 05 2017, @07:49PM
Ah that's just vSphere. Other than having multiple almost identical appearance web UI, vSphere isn't all that bad. The cobbling is more related to licensing. You can buy a system without disaster recovery automation so there are aspects to openstack that are baked into the cake for restoral of a dead host that have to be kinda grafted onto vSphere and can't be the default. From memory you turn on DRS migration at the cluster level whereas on openstack there are no pay options so its kinda enabled by default AFAIK.
Another weird example is openstack has "neutron" the network mismanager with everything baked in and you can't really do vmware "standard ethernet switches" with openstack, AFAIK, because openstack only offers distributed switching but you can buy ESX hosts without vSphere to manage distributed ethernet switches so being licenseable it has to be bolted on. If you're doing virtualbox or just screwing around on linux, vmware "standard ethernet switches" are just linux bridge networking, and what you configure has no effect on the config of other hosts. Distributed switches are more than just automation there's some weird shared uplink routing stuff going on that I haven't explored.
Or in summary, ironically because vmware isn't free, there are options to purchase for networking that make networking more complicated for vmware than for openstack, where everythings free, so why would anyone ever use anything but distributed ethernet switches on simpler openstack?
Yeah that's it pretty much. I'm a bit fuzzy on it myself. There is an API to mess with vmware stuff. Maybe that API is part of vRealize LOL.
(Score: 2) by kaszz on Monday June 05 2017, @11:50PM (1 child)
Without proper docs it's possible to charge more for support calls..
(Score: 2) by VLM on Tuesday June 06 2017, @03:48PM
In theory true but even if generally true there's a spectrum of it and Cisco for example used to have legendary good docs online AND I was personally involved at a company where Cisco Certs meant a price deduction on the huge bill, a deduction big enough that paying me to shitpost and play video games all day would still have been a net profit to the company. Of course rather than shitpost and video I got stuck providing BGP support to our customers, which I was very good at although it got boring after the 50th time some guy tried to send us a 0/0 route or wanted to advertise his previous ISP's subnet, LOL. "Well see I was tryin to redistribute my RIP routes into BGP" "Excuse me sir WTF are you doing, STFU and configure it the way I tell you to" (although I spent about four hours per customer or so it seemed doing it very politely, although the previous one liner summarized it remarkably well). I also felt like a hostage rescue negotiator some days "I see my routes are dampened because of flapping, what does that mean, I guess I'll go reboot my router a couple times till it starts working again" "nooooooooo nooooo don't jump your router has a lot to live for nooooooo"
(Score: 3, Funny) by DannyB on Monday June 05 2017, @03:44PM (12 children)
Good documentation has been around for decades too.
But it was three feet thick. And bolted to a table. Documentation could not physically be moved to another table, let alone taken from the computer room. You had to memorize three feet of documentation. There was no GUI. It was uphill both ways. Hey you kids, get off my lawn!
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by VLM on Monday June 05 2017, @05:20PM (11 children)
Like you I miss the pre-internet days.
I think you're mixing your memes, I had momentary access to VMS and a DEC wall and a DEC wall was about 3 feet of color coded binders and supposedly it made sense after you memorized all three feet. All I remember was the CLI for VMS was something like DCL DEC control language or something and it had a fabulous CLI with respect to consistency and online help. I had much longer term access to sunos (like pre-solaris unix) and hp--(s)-ux and for one of them the machine room did have a users manual that boiled down to a printout of all the man pages in the form of a 4 inch thick phonebook and it was on a stand bolted to the machine room desk console. Supposedly that manual cost $1000 or so they told us, probably to stop us from ripping out pages.
What I miss about pre-internet days was when IBM or DEC shipped you a set of binders, thats honestly all you needed to know. You didn't have to google and find 200 sources and figure out which are clickbait and which are real.
If you had a modern 90s CMOS mainframe from IBM I forget the name but I remember reading an entire ATM training manual of a couple hundred papers in an emulated 3270 terminal on roughly a 386 in about '95 or '96. I've never experienced anything quite like it since. I've seen the occasional "here's a pdf best of luck" as docs but IBM had amazing detailed manuals that all worked together.
As for retro stuff, DECs PDP8 book series of manuals was simply amazing never seen anything like it since from a single source. Imagine all of 80s home computer tech times 100 or so all single sourced. Great manual series.
Some other retro DEC stuff that was decent was the TOPS-10 manual series. Supposedly TOPS-20 was a better OS with inferior docs. TOPS-10 was a cool OS.
(Score: 2) by DannyB on Monday June 05 2017, @05:53PM (10 children)
Back when I was a snot nosed 20 year old kid, I was able to brain download manuals. Much later, in about 1986 (as a Mac developer) I discovered Lisp. There was this XLisp I could download from CompuServe. Later I spent $495 (in 1989 dollars) on Macintosh Common Lisp. Used it until about 1993. Loved it. Still have every line of code I wrote. (Now I like Clojure.) I had bought my own copy of CLtL and CLtL2 [cmu.edu]; and brain downloaded those. They were well thumbed.
For the youngsters, CompuServe is something from a different millennium. Before web sites. Before dial up internet. Before Usenet was popular. Even before AOL. Back when modems were 2400 bits/second as God intended.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by VLM on Monday June 05 2017, @06:19PM
Noobs. My dad and grandfather has TRS-80s with 300 baud modems. Eventually my dad upgrades to a TRS80 4p with internal 1200 baud modem, which was pretty fast. Radio Shack used to include like "free 5 hours of compuserve" little carbon paper free accounts with all manner of products.
Believe it or not I was BBS-ing in '87 or '88 with a 1200 baud modem I got from the "free" bin at a local ham radio fest (back in those days, radio fests had a lot of computers). I believe in Christmas of '89 I got a 2400 baud modem, doubling my speed was quite nice. That served until I blew like $500 in '92 or '93 and got a 14.4K and around then internet access was becoming a thing so I got a SLIP account with a dedicated static address, pretty cool.
(Score: 2) by kaszz on Tuesday June 06 2017, @12:06AM (8 children)
Which of the LISPs are worthwhile btw? as in being powerful to do whatever one might wish without over complicating matters?
* Erlang
* Common Lisp
* Clojure
* Haskell
Compuserve was like a BBS that were powered by mainframes connected worldwide right?
Usenet.. killed by eternal autumn and lawyers peddling green cards. A lot of the big hosts handling it went away and others could not pick it up because the share volume and processing involved. But now capacity is cheap so maybe a revival is possible?
(Score: 2) by hendrikboom on Tuesday June 06 2017, @02:08AM (6 children)
There are two interesting Lisps; both of them are Scheme implementations.
(1) Racket: Racket's specialty is multilingual programming. You can define new syntaxes and new semantics (i.e. new languages) and compose a program from modules written in those languages. The feature seems to cover everything from minor tweaks to completely different languages. For example, you can write modules in Algol 60 or in Scribble. Scribble is a notation for Scheme that looks and feels like a text formatting markup language.
And Racket has excellent tutorials and documentation and a supportive mailing list, to return this discussion to the original subject.
(2) Gambit. Gambit is a Scheme implementation that compiles directly to C (or C++) (Or you can just use the interpreter). Its virtue is that you can actually introduce new features and specify just what C code is to be generated from it. As a bonus you get a rather flexible scripting language. And you get to use low-level Cisms when you want.
-- hendrik
(Score: 2) by kaszz on Tuesday June 06 2017, @03:20AM (5 children)
Do any of these LISPs make efficient use of multi cores that are common now, but were not when those languages came into being? They should be able to exploit that even better than many other languages simple by their nature.
(Score: 2) by DannyB on Tuesday June 06 2017, @02:17PM (1 child)
I would agree that Racket and Gambit are interesting. If you are interested in Scheme. There is also Chez Scheme, which was commercial, but then became open source.
https://en.wikipedia.org/wiki/Chez_Scheme [wikipedia.org]
http://www.scheme.com/ [scheme.com]
https://github.com/cisco/ChezScheme [github.com]
Incremental native-code compilation to produce native binaries for the PowerPC, SPARC, IA-32, and x86-64 processor architectures. Chez Scheme supports R6RS.
Chez Scheme has a windowing and graphics package called the Scheme Widget Library, and is supported by the portable SLIB library.
I like Clojure. (clojure.org) Clojure is a modern lisp. It has reach, it runs on: JVM, JavaScript, .NET. Clojure runs on JVM and .NET, while ClojureScript compiles to JavaScript and runs in-browser or other JavaScript implementations.
Momentary sidetrack on JVM (Java Virtual Machine) . . .
JVM is the runtime engine that runs Java Bytecode which is emitted from the Java compiler, Clojure compiler, and other language compilers (Kotlin, Scala, Groovy, Jython, JRuby etc).
JVM is interesting because it is an industrial strength runtime platform. A decade and a half of research has gone into JVM's JIT compilers C1 and C2, and it's multiple GC's. When you run JVM you can select "server" mode or "client" mode (eg, tune it for running on a workstation or on a server). You have a choice of GC algorithms to pick from, and gobs of GC tuning parameters. If you simply give it plenty of RAM, JVM is very fast. The GC's run in parallel on multiple CPU cores. So if you throw plenty of CPU cores at it, you may never see any GC pause times.
You can get multiple implementations of JVM. Oracle. Or Azul's Zing -- which is a free binary build (with source) based on the open source OpenJDK. (Also Oracle's Java is based on OpenJDK) If you're on IBM mainframe hardware, then IBM provides Java runtime. Java runtime is available on Raspberry Pi. Languages that compile to JVM can be used for Android development (which compiles JVM bytecode into Dalvik bytecode to run on Android).
Java (or rather JVM and various languages) are popular in enterprise software. Java is used for high frequency trading (yes really).
If you need a JVM that runs on hundreds of GB of RAM, and with up to 768 cpu cores, then look at Azul Zing on Linux.
We now return to Clojure . . .
Without being grandiose, Clojure runs on an ordinary desktop java runtime, even on a humble Raspberry Pi.
Clojure is functional programming.
All data in Clojure is immutable. That means you can't alter anything in a data structure. If I have an array of 50 million elements, and I change element 1,253,507; I get back a new array with that element changed. The original array is unaltered. Yet the performance guarantee is "close to" what you expect of arrays for indexed access. How is this magic implemented behind the scenes? With 32-way trees. When you changed element 1,253,507, a certain leaf node in the tree was changed. That new node, along with new nodes all the way up to a new root node become the new array. The tree shares structure with all other tree elements of the old array. Thus only a few tree nodes are recreated at a cost of Log32(n). So it's close to direct indexed performance for the huge benefit of immutable data structures. This means there is no such thing as: (SETF (CADR x) 'foo) That would be trying to surgically modify a data structure. There are similar operations that can do this, but they re-construct the portions of the data structure, which ultimately makes use of the underlying (invisible to you) implementation of immutability.
All variables are final. (Meaning, once a variable is assigned a value it cannot be changed to a different value.)
This may sound restricting, but Clojure definitely has the right constructs to give it a functional style that mitigates what you might think of as restrictions. The end result is better quality code. You can see Haskell inspiration.
Clojure has primitive types for Lists, Arrays, Sets and Hash Maps. Lists are what you already know. Arrays are what you expect. You can alter any element in the array by direct index, far cheaper than in a list, but with the guarantees of immutability of the original array (you get back a new array with the single element altered). Sets are what you expect. Sort of like an array or hash map, an item can only occur once in the set. Hash Maps are key-value stores like in many modern languages, but backed by an implementation that uses different backing for different sized hash maps. A small hash map of a dozen items will not use hashing, but merely be internally an array where the key array is searched for the key. At some invisible magic threshold the underlying implementation becomes a hash map. Clojure has lots of hidden implementation optimizations, like for short lists, arrays, etc. Implementations of functions with certain common parameters.
Clojure has excellent Java interoperability (when used on Java). Think of this like other lisps having an FFI to C. Clojure and Java (or other JVM language) code is fully interoperable. Clojure data structures can be passed to Java and manipulated or even created in Java by importing the Clojure imports into Java code to be able to access the Clojure data structure APIs. Java data structures can be passed to Clojure and accessed with dot notation to access methods and class members. You can write JVM classes in Clojure for compatibility with other Java code. For example, if using a Java library that needs to be passed a callback function, it is easy to write this callback function in Clojure without using any Java.
Because it runs on JVM, Clojure has access to the EMBARRASSINGLY large amount of riches of Java libraries. Code to do anything under the sun. Including access to the GPU. And there are Clojure libraries for working with the GPU.
Clojure has an excellent story about concurrency. I won't go into it here. But you can write high level code that will run on all your CPU cores. In Clojure, MAP is like MAPCAR in CL. Instead of using "map", you can use "pmap" to process concurrently if you don't care about the order of the returned elements. (eg, I have a list of ten million items, and I need to run function F on all of them, I can use (pmap F items) if I don't care that the returned list is in a different order than items. Watch all your CPU cores light up.)
Clojure has a nice syntax for Arrays, Sets and HashMaps. Thus you can render arbitrarily complex data structures to text, and then back to internal data again. Think JSON but much better.
You'd be amazed with what people do in Clojure because of Java interop. Playing with MIDI and synthesizers, even software synthesizers. See Clojure's interface to Overtone. Using Clojure with OpenCV (computer vision).
Let me throw out another interesting one.
Pixie Lisp
It compiles to native code. Built with the Rpython tool chain. Has good C FFI because of this. Early implementations on Raspberry Pi with direct Lisp access to WiringPi which provides access to the hardware GPIO pins (digital / analog / SPI / I2C / PWM input and output pins).
Pixle lisp is also Clojure inspired.
Finally, let me mention: Shen, a sufficiently advanced Lisp.
(to be indistinguishable from Magic. See that it is like having Haskell and Prolog baked into the Lisp itself.)
Hope that helps.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by DannyB on Tuesday June 06 2017, @02:19PM
Duh. "Or Azul's Zing -- which is a free binary build" I meant Azul Zulu.
Zing is the one that runs on hundreds of GB of ram with up to 768 cpu cores.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by VLM on Tuesday June 06 2017, @03:59PM (1 child)
There are no obvious factual errors in Danny's reply to the best of my knowledge at this time.
A very short answer to your very specific question in a limited sense is "yes"
A longer answer to that specific question including plenty of low level implementation examples is at:
http://www.braveclojure.com/concurrency/ [braveclojure.com]
Clojure for the Brave and True is a bit of an acquired taste but I figured it would be more fun than a link into clojuredocs.org. clojuredocs is best used if you already know you want a future and you merely forgot some detail about futures.
Its not a magic or magic-ish language like Erlang you kinda have to intentionally parallelize stuff in Clojure for it to run multi-core but its not hard either and there's plenty of support.
It works.
(Score: 2) by DannyB on Tuesday June 06 2017, @04:39PM
This [objectcomputing.com] is an intro that I found useful as an into to Clojure, a long time ago.
I think I remember an amusing YouTube video where Rich Hickey mentions running some cool core.async program on Azul's custom hardware with hundreds of cpu cores.
I am excited to try Pixie Lisp on a Pi. So many projects to tinker with. So little time. Ugh.
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 2) by hendrikboom on Saturday June 10 2017, @01:22AM
I believe Gambit has a multicore implementation in beta testing -- including multicore garbage collection, But not garbage collection running in parallel with program execution.
Racket may well be working on something similar, but I don't remember the details.
-- hendrik
(Score: 0) by Anonymous Coward on Tuesday June 06 2017, @10:04AM
Might be minus the -.
What killed usenet wasn't the volume per-se, it was the BINARIES being pushed to newgroups, even non-binary usegroups once ISPs started filtering those. There was a chart up a few years back showing usenet daily capacity from the 80s to the early-mid '00s. The content amount went from a few gigabytes a day to tens of gigabytes a day, to a few hundred gigabytes a day, and by the '00s, was into the terabytes a day of traffic volume. Yes. Terabytes. In an era when drives were still sub-terabyte, and backbone capacities were probably below... OC48(?) with ISPs having OC3 or competitive being reasonably high end for non-corporate local ISPs.
THAT, combined with spam is what killed Usenet. It could be reestablished today, but there are fundamental changes to the protocol that should be made to help synchronize mirroring of chains of posts, modifications rather than reposts for changed posts, and better peering of new blocks of messages (nntp was point to point, which is reponsible for a lot of the traffic overhead. It was okay in the UUCP era, but nowadays something bittorrent-esque for helping spread publishes across multiple fast hosts could help immensely.)
(Score: 2) by Wootery on Tuesday June 06 2017, @04:14PM
Surprised no-one seems to have mentioned this, but I suspect a major reason for bad documentation of FOSS is boredom.
To be sure, plenty of FOSS is written by paid developers, but lots of it is written 'for fun', and writing documentation isn't something many programmers enjoy doing.
I don't see any reason to jump to the conclusion that FOSS developers can't write documentation.
(Score: 4, Insightful) by fraxinus-tree on Monday June 05 2017, @12:30PM (3 children)
I hate to say it, but a lot of them don't know how to write code, either.
(Score: 2) by turgid on Monday June 05 2017, @12:38PM (1 child)
Indeed, and that observation applies to software in general. 99% of programmers don't know how to write code. They're "unconscious incompetent." At least I'm "conscious incompetent" and trying hard to improve. Arrogance and hubris seem to be the order of the day. I weep for the future of the Human Race.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 0) by Anonymous Coward on Monday June 05 2017, @08:01PM
i can agree with this. i've been learning a language for a couple years(?) now and i can make things that that are well above my pay grade. i am well aware that i have much to learn but i can see evidence everywhere that many people think that they excrete petunias.
(Score: 2) by DannyB on Monday June 05 2017, @03:47PM
These "learn programming in 24 hours" type books need to be: Learn to program in only ten years!
It's not that you're not writing code right away. It's that you don't really know what you're doing. With experience comes the "what was I thinking when I wrote this?!?"
Don't put a mindless tool of corporations in the white house; vote ChatGPT for 2024!
(Score: 5, Interesting) by VLM on Monday June 05 2017, @01:03PM (2 children)
And whats the literacy level of the target population?
I know the literacy level of the "hacker" "techie" community is enormously higher than the normie community but its still not that high.
Anecdotally I remember cracking open my first electrodynamics textbook at uni I donno maybe it was some old edition of Griffiths, whatever, I can assure you it seemed "incomplete or confusing documentation" of how the universe works. Yet with some study I passed the class... You want to do crazy high level wizardry, well the spellbook for that is naturally going to look a little mysterious to the first time viewing apprentice. Maybe, if you don't know what a primary domain controller is, you shouldn't be setting on up on the prod network at work. Maybe if you don't know what BGP is, you shouldn't be in charge of configuring the router that connects to 4 upstream ISPs.
Note that people who are operating at the correct level will not be served by scrapping the higher level spellbooks and replacing everything with "see spot run" level stuff. Thats the classic Linux desktop environment mistake. I don't want to use a Linux designed for people who barely know how a mouse works, any more than a master craftsman fine finish carpentry expert wants to use a plastic pink "hello kitty" hammer instead of pro tools.
Anyway there's two other comments. If you want to understand electromagnetism and all that, there's the classic text "Div, Grad, Curl, and All That: An Informal Text on Vector Calculus" which is kinda a pre-req no matter if you know it or not. And there's books like the ARRL handbook and numerous other ham radio texts which will give you a sort-math-free education about RF which will help a lot when you're learning the math. Just saying if you've never taught yourself anything, there are probably dozens if not thousands of sources so if you're reading a particularly bad source, try another source.
I'm reading a sci fi book called ninefox gambit but in the mostly non-fiction category I'm reading "Dispatches" by Mike Herr which is the autobiography of a boomer "journalist" who covers the Vietnam War while basically being high all the time. its kinda like Hunter S Thompson goes to the Vietnam War as a correspondent instead of joining the Hells Angels. Dispatches was written about ten years after Hells Angels and there is some resonance. Anyway as a Vietnam War documentary its shit compared to a book like "Fire in the Lake" its totally "incomplete or confusing documentation". Yet it still serves its purpose. If you want a documentary, and pick up something else, that doesn't mean its a shit book or you're illiterate, it just means you're really bad at selecting books for a given topic. If your book sucks at task "X" then ask someone what book to read for task "X". Or a more concrete example lets say you hate fantasy and love sci fi and someone trolls you into reading the LOTR trilogy, you're gonna hate it, and that doesn't mean the book sux or that you can't read, but it does mean you can either get trolled or aren't good at self selecting books to read.
On the topic of gonzo journalism I wonder if I could drop a lot of acid and smoke pounds of weed and write something like "the gonzo guide to freebsd" with lines like "VLM knew he was in trouble when the install flash drive he was holding seemed to change shape into a frog and hopped away". People like books like that about the Vietnam War or Hells Angels or Hippies so why not installing FreeBSD?
Something to think about is once workplaces settle down the documentation always improves. Very few 1st year carpenter's apprentices are handed a structural engineering statics book either accidentally or on purpose. Someday, when its settled what a noob is and what a noob does, they'll be some pretty good noob books out there.
(Score: 1, Interesting) by Anonymous Coward on Monday June 05 2017, @03:16PM (1 child)
Before it was called "Gonzo" there was Normal Mailer's "The Naked and the Dead" -- this sets a very high bar for the "hyper-realistic war documentary fiction" category, which I believe Mailer invented(?), although in other writings Mailer says that he was inspired by Faulkner...
Tom Wolfe and Hunter Thompson surely read Mailer when they were starting out.
---
> flash drive he was holding seemed to change shape into a frog and hopped away
This reminded me of a trippy college evening. I got back to the co-op where I was living and discovered that it was my day for kp. Had to wash the dishes for 30 people, while the dishes were grinning at me and and trying to jump out of my hand and onto the floor. I focused hard and none of them escaped!
(Score: 2) by VLM on Monday June 05 2017, @03:41PM
Interesting I donno how I missed that book...
I'm very familiar with Leckie "Helmet for my pillow" and Sledge "With the old breed" and its interesting they didn't write their books until decades after the war.
A lot of gonzo seems to be of the "alienated journalist is LARPing as not really a journalist not really a participant and high on drugs most of the time", I wouldn't say "Dispatches" is very realistic portrayal of life as either a REMF or a front line grunt, its a realistic portrayal of kind of a journalist/tourist. Toward the end of "Dispatches" the author gets super alienated because he's not even a real journalist because he's writing for a monthly magazine and "real" journalists either write for daily newspapers or are weekly photojournalists so he spent an inordinate amount of time baking and goofing off. I kind of like the book "Dispatches" but I get the feeling the author himself is (or at least was as a kid) a degenerate piece of shit, none the less with some writing skill.
Now, Sledge and Leckie were front line grunts, and tell quite a story. Also they're likable unlike the poseurs Thompson and the "Dispatches" author.
(Score: 3, Interesting) by hendrikboom on Monday June 05 2017, @08:12PM
I've started predocumenting. I start to explain what I'm trying to do and how I'm trying to do it in English. It's often quite vague, but it shows the approach I'm taking, and after a while, also shows the approaches I'm not taking. It's only after I do this that I start writing code. Sometimes the predocumentation ends up as comments in the code, sometimes not.
And I always have a lot of TODO comments -- places where I'm not quite sure I've done something the best way, or even an adequate way. Sometimes I know it's not the right way, but I know the code will enable me to write and test other parts of the code. That gets a TODO. Sometimes i know what I wrote won't scale up. That gets a TODO. The TODOs look very informal, and that is the right level of precision.
Of course the code doesn't match the predocumentation. For one thing, the predocumentation contains the approaches I could have used as well as the one I finally ended up using. And a battle plan doesn't survive contact with the enemy. But I often find that what I planned to do doesn't work long before I end up debugging. That saves me a lot of work.
At all times, the code and the associated documents present what actually there and what should be there, incompletely and approximately. The writing doesn't look definitive. I find it useful a year later when I'm looking at the code again. It tells me what on earth I was thinking when I wrote it.
Anytime the code confuses me when I come back to it, I figure out what's going on and document it. Or document what it should have been and make the necessary changes.
(Score: 0) by Anonymous Coward on Tuesday June 06 2017, @05:03AM
I'm surprised everyone left out another reason for no docs these days: many of the developers can't really speak English. They know their English is so bad, they don't bother writing any.