SoylentNews is people
SoylentNews
The growth of command line options, 1979-Present
The first of McIlroy's dicta is often paraphrased as "do one thing and do it well", which is shortened from "Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new 'features.'" [ . . . ]
If you open up a manpage for ls on mac, you'll see that it starts with
ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...]
That is, the one-letter flags to ls include every lowercase letter except for {jvyz}, 14 uppercase letters, plus @ and 1. That's 22 + 14 + 2 = 38 single-character options alone.
On ubuntu 17, if you read the manpage for coreutils ls, you don't get a nice summary of options, but you'll see that ls has 58 options (including --help and --version).
To see if ls is an aberration or if it's normal to have commands that do this much stuff, we can look at some common commands, sorted by frequency of use.
(see article for interesting table.)
We can see that the number of command line options has dramatically increased over time; entries tend to get darker going to the right (more options) and there are no cases where entries get lighter (fewer options).
Everything was small and my heart sinks for Linux when I see the size [inaudible]. The same utilities that used to fit in eight k[ilobytes] are a meg now. And the manual page, which used to really fit on, which used to really be a manual page, is now a small volume with a thousand options... We used to sit around in the UNIX room saying "what can we throw out? Why is there this option?" It's usually, it's often because there's some deficiency in the basic design -- you didn't really hit the right design point. Instead of putting in an option, figure out why, what was forcing you to add that option. This viewpoint, which was imposed partly because there was very small hardware ... has been lost and we're not better off for it.
[ . . . . ] Another reason commands now have more options is that people have added convenience flags for functionality that could have been done by cobbling together a series of commands. These go all the way back to v7 unix, where ls has an option to reverse the sort order (which could have been done by passing the output to tac).
Over time, more convenience options have been added. For example, to pick a command that originally has zero options, mv can move and create a backup (three options; two are different ways to specify a backup, one of which takes an argument and the other of which takes zero explicit arguments and reads an implicit argument from the VERSION_CONTROL environment variable; one option allows overriding the default backup suffix). mv now also has options to never overwrite and to only overwrite if the file is newer.
mkdir is another program that used to have no options where, excluding security things for SELinux or SMACK as well as help and version options, the added options are convenience flags: setting the permissions of the new directory and making parent directories if they don't exist.
[ . . . ] unlike with a GUI, adding these options doesn't clutter up the interface.
(emphasis added)
No worry, systemd will guide us back to the true Unix Microsoft way.
(Score: 5, Insightful) by theluggage on Wednesday March 04 2020, @11:37AM (8 children)
As long as all those options are concerned with creating a text-based list of the files in a particular directory, formatted in various ways, so what? In what bizarro-world alternate universe would it make sense to maintain separate programs for long-format lists, shot-format lists, date-sorted lists, long-format date-sorted lists...?
OK, you could structure it as a 'raw' machine-readable listing generator piping in to various formatting modules - but then its very inefficient to have to generate all the possible file data for simple formats so you'd need the generator module to have options to filter the data in various ways and... Oh, wait...
The problem comes when some bright spark says "Hey, could we have an option where if you 'ls' a single file it 'cat's it instead?", "How about having a system of plug-ins based on a subset of 'awk'?" Or, that modern classic, "We need an option to share your directory listing on Facebook?"...
Thing is "McIlroy's dicta" may be a good idea, but such rules are only there to make you think before breaking them. The idea that you can turn good practice guidelines into simple by-the-numbers metrics is real MBA-level thinking.
(Score: 2, Insightful) by Anonymous Coward on Wednesday March 04 2020, @11:55AM (1 child)
No the problem is when seventeen different Linux implementations decide they need seventeen different sets of command line options.
(Score: 0) by Anonymous Coward on Wednesday March 04 2020, @01:38PM
Is there anything stopping a person from using tac instead of the ls option to reverse the list?
Is there anything forcing people to know the full list of options?
Is the complaint simply that the programs are getting larger? How many people care about larger versions of these programs when the rest of everything is so much bigger?
(Score: 3, Interesting) by FatPhil on Wednesday March 04 2020, @02:32PM (5 children)
In some sense "the way" is the *single* choice that needs to be selected, where many options to select from. The problem with that is it doesn't work well with short options.
However, gcc happily offers -O1, -O2, -O3, -Os, or whatever, so unix people should be used to the condensing of selections into a single switch.
I think my biggest 2 complaints are:
- inclusion of an arbitrary set of compressors into tar. If pipes are too slow, fix pipes - that helps *everyone*. If your OS vendor refuses to fix pipes, maybe you're buying your OS from the wrong vendor.
- grep is now twice the size of sed, despite the fact that sed can do way more than what grep was designed to do:
luser@cubispaz:/usr/local/homes/luser$ size `which grep`
text data bss dec hex filename
205149 4312 4776 214237 344dd /bin/grep
luser@cubispaz:/usr/local/homes/luser$ size `which sed`
text data bss dec hex filename
100142 2724 2480 105346 19b82 /bin/sed
I genuinely don't think I could name more than about 1/5th of the switches to any of the programs in his list now, some of the counts were shocking. I've been using unices since the late 80s, and whilst I have grown into some of the new features, I frequently come across things that seem to be added for no good reason at all (which I promptly forget, as I will never have any use for them).
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Interesting) by NotSanguine on Wednesday March 04 2020, @04:28PM (4 children)
For the most part, like the author of TFA, I'm not all that annoyed (except for the out-of-proportion file size increases) with the proliferation of command line options (from TFA):
Not because I use all those command-line options, but because some the added ones are useful and don't make tools harder to use.
That said, I agree with OP about the inclusion of compressor/decompressor command line options to tar.
Why should some arbitrary set be included while others aren't? What's more, pipelines provide extreme flexibility for this:
tar cf - foo |bzip2 -9 > foo.tar.bz2
bunzip2 -c foo.tar.gz | tar xf -
Just replace bzip2 above with the compression too of your choice.
I suppose a '-zo </path/to/compressor>' option could be generally useful, but all that does is add complexity to tar and doesn't really simplify the task.
I guess I'd say that as long as the addition a command line option doesn't limit functionality, who cares?
No, no, you're not thinking; you're just being logical. --Niels Bohr
(Score: 1, Interesting) by Anonymous Coward on Wednesday March 04 2020, @10:14PM (3 children)
There is a major deficiency with shell pipelines. Only the exit status of the last command is returned, the exit status of all other commands are silently ignored.
Consider the example given (I will note that specifying a file of - is redundant, tar will write to standard output by default):
So if "tar" fails to create the archive—for example, because there was an error reading some file—then the caller does not find out about this. The compressor is unlikely to fail so this command basically always returns a successful exit status. The tar program is very frequently used from scripts and if you write a command like this in a backup script you might not realize until too late that half of your documents have not been backed up for the past 6 months.
Some advanced shells provide a way to access the results of all commands in a pipeline. You probably forgot to use them.
If you need to get the exit status portably (say, for example, you want a script to run on any POSIX-ish shell) getting the exit status of commands in the pipeline is still possible but quite complex. You probably forgot to test some edge case.
Murphy will get you every time.
With the integrated compressors we can just write something like tar czf backup.tar.gz files || { echo 'BACKUP FAILED'; exit 1; } and it will work.
(Score: 2) by NotSanguine on Wednesday March 04 2020, @11:51PM (2 children)
If I'm not scripting, it's irrelevant, as I'll confirm the results manually.
If I am scripting, something like
tar cf - foo 2>script.log |bzip2 -9 > foo.tar.bz2 2>>script.log
If the process is automated, there should be a log, and a responsible party (or a script) should be checking that log, no?
As such, something similar should do the trick.
Alternatively, using the integrated compressor would work just fine too.
That's one of the nice things about *nix, there's never just *one* way to do things. If I want to use a compressor that's not supported by tar, I can do that too, if I want.
No, no, you're not thinking; you're just being logical. --Niels Bohr
(Score: 0) by Anonymous Coward on Thursday March 05 2020, @09:01PM (1 child)
This (or a variant thereof) is a reasonable approach to solve the problem. In fact the "portable posix shell" way of getting the exit status of earlier commands is essentially to do just that -- each command in the pipeline must squirrel away the pass/fail status somewhere, so that the script can check the command status afterwards.
In the above pipeline, the script.log file is opened twice: once in regular mode and once in append mode. This is going to produce interesting (i.e., corrupt) results if both commands print errors because the file position is not shared between the two processes.
The above pipeline as written will not record any error whatsoever if "tar" is killed by a signal, so there will be no way to detect if that happened by looking at the log file.
Dealing with all of this makes scripts substantially more complex than just using an integrated compressor. Since tar is very frequently used with compressors, it makes a lot of sense to integrate commonly-used compressors into "tar" itself so these problems can all be solved in one place (in tar) rather than having to deal with this issue in every single script that ever uses tar.
And that is why I consider integrated compressor options to be a good addition to the "tar" utility.
(Score: 2) by NotSanguine on Thursday March 05 2020, @10:22PM
I shan't disagree, as you make lots of sense.
Rather I'll just reiterate that it's good that there's always more than one way to do things in *nix.
No, no, you're not thinking; you're just being logical. --Niels Bohr