We make very careful considerations about the interface and operation of the GNU coreutils, but unfortunately due to backwards compatibility reasons, some behaviours or defaults of these utilities can be confusing.
This information will continue to be updated and overlaps somewhat with the coreutils FAQ, with this list focusing on less frequent potential issues.
Good tips and reminders for those who don't work mostly with a CLI (Command Line Interface).
[What has been YOUR biggest CLI gotcha? -Ed.]
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(Score: 2, Informative) by Anonymous Coward on Tuesday December 01 2015, @07:58PM
Wonder if RedHat requires employees blog 1000 characters a year or something.
Ease of use: who doesn't use an ls alias to turn on colour?
*I* do not use an ls alias to turn on colour, and I can't stand systems that make it the default.
Nothing like dark blue on black to make you want to set fire to your terminal.
(Score: 1, Insightful) by Anonymous Coward on Tuesday December 01 2015, @08:06PM
Sorry, was from another one of this dude's pages--forgot the link: http://www.pixelbeat.org/docs/power_of_the_default.html [pixelbeat.org]
While I'm at it:
tail -F is probably what you want rather than -f as the latter doesn't follow log rotations etc.
That's a "Gotcha?" I've been using "tail -f" for about two decades and only needed to tail a rotated log file once, ever. And the man page tells you exactly how to do it.
(Score: 0) by Anonymous Coward on Tuesday December 01 2015, @08:38PM
Hear Hear!
Ugh....dark blue on black....the first think I do after setting up a new system is edit /etc/DIR_COLORS and change that to yellow.
(Score: 3, Informative) by Anonymous Coward on Tuesday December 01 2015, @09:10PM
You haven't switched to Solarized [ethanschoonover.com] yet?
(Score: 2) by morgauxo on Tuesday December 01 2015, @10:05PM
I always turn on color. I've never seen results where the colors don't show up well against one another. Do you have some sort of old custom color settings sitting in your home or etc directories and overriding the default colors?
I can't stand distros where they are not on by default. Actually.. that's too strong... I think it's far more attractive with them on and distros should default them to on. But.. I suppose I could stand it if I had to. What I can't stand is that default .bashrc comment that says they are off because they are 'distracting'! What a loaded, subjective statement! I don't think they are distracting at all! A screen full of nothing but plain white text... to me that is distracting. Maybe not to you... They don't need that comment there.
(Score: 2) by Subsentient on Tuesday December 01 2015, @10:27PM
Same here. Colors make ls a lot more useful for me. I often just want a brief listing of files, not ls -l, and the color adds some information I can use.
"It is no measure of health to be well adjusted to a profoundly sick society." -Jiddu Krishnamurti
(Score: 0) by Anonymous Coward on Wednesday December 02 2015, @01:00AM
Ugh. The out-of-the-box Mint puts dark blue on black for me. If you don't change it, you need to put your nose right on the screen to read the text.
(Score: 2) by lentilla on Wednesday December 02 2015, @04:31AM
I too will agree. I like ls to use color by default.
My reasoning: a) it adds semantic information to what would otherwise be a homogeneous wall of filenames; b) you can turn it off it you don't like it; and; c) it lets new users know the colorization "feature" exists in an relatively unobtrusive way.
Like many others, I've come across ls colorizing using inappropriate colors for my terminal. (Probably happens more often to me: each new xterm I open uses a different foreground/background color scheme. My monitor looks like a candy shop and I don't loose track of which xterm is which as easily.) Whenever ls' colors don't work I smile and do a quick work-around because I'm grateful for the other 99% of the time they work well.
(Score: 1, Insightful) by Anonymous Coward on Tuesday December 01 2015, @08:00PM
If two or more hard links point to the same file, only one of the hard links is counted.
Not sure that's a bad thing or not. Sounds like desirable behavior in some instances.
(Score: 1) by xav on Wednesday December 02 2015, @03:29AM
Absolutely. I use this feature on a regular basis to compare two backups of the same directory (hard link backups with rsync).
"du -sm dir.day-2 dir.day-1" tells me what amount of data has changed between the two backups.
(Score: 2) by bart9h on Wednesday December 02 2015, @01:36PM
Of course it's a good thing. 'du' = 'disk usage', that is, the amount of disk space that is occupied by the files, not the sum of each file size.
(Score: 0) by Anonymous Coward on Tuesday December 01 2015, @08:31PM
# rm -rf /tmp/crap / tmp/shit
(Score: 4, Funny) by termigator on Tuesday December 01 2015, @08:42PM
Remove those pesky dot directories:
# rm -rf .*
Oops!
(Score: 0) by Anonymous Coward on Tuesday December 01 2015, @08:44PM
# rm -rf .*
Ouch!
(Score: 2) by KilroySmith on Tuesday December 01 2015, @09:14PM
Yeah, I did that once.
In my first week at a new job.
In the root of the Netware server that we were using which had been shared for "convenience".
I sat there wondering "Why is this command taking so long", and as I stared at the command line, and realized what I'd done, the other developers around the room started asking "Hey, what's up with the server?".
Ouch.
(Score: 0) by Anonymous Coward on Wednesday December 02 2015, @01:52AM
That "Why is this command taking so long" is the WORST FEELING IN THE WORLD. You go from zero to absolute terror in about 3 seconds, then it's ^C^C^C^C^C^C^C^C^C^C time.
I feel your pain.
(Score: 2) by tibman on Wednesday December 02 2015, @04:15AM
Chest pains.. HNNNNNG
SN won't survive on lurkers alone. Write comments.
(Score: 3, Funny) by termigator on Tuesday December 01 2015, @08:47PM
> What has been YOUR biggest CLI gotcha?
Having to work with Windows DOS console and .bat scripts.
(Score: 2) by el_oscuro on Tuesday December 01 2015, @10:37PM
While The Windows command prompt is a kludge, it is surprisingly powerful and has an excellent man page with lots of great examples on how to use it effectively.
I know it is very obscure, but the command to access the man page is "help". Who would ever think of that?
SoylentNews is Bacon! [nueskes.com]
(Score: 3, Informative) by termigator on Wednesday December 02 2015, @12:14AM
But the parsing rules regarding quoted content and variable expansion is insane.
I have the unfortunate task of maintaining DOS script wrappers for a project, and trying to do things I can do easily in *nix-based shells is an experience I would like to forget.
(Score: 0) by Anonymous Coward on Wednesday December 02 2015, @01:16PM
(Score: 1) by shipofgold on Tuesday December 01 2015, @08:54PM
I always like the philosophy of Rob Pike as detailed at:
http://harmful.cat-v.org/cat-v/ [cat-v.org]
I still like commands with few options to which you can send output to other filters to get what you need. tar never needed the -x or -j or -J options.
My favorite gotcha is the difference between the for loop and the while loop in bash. While loops create subshells with a new environment while for loops do not. Makes changes to bash variables very interesting.
(Score: 3, Informative) by draconx on Tuesday December 01 2015, @09:10PM
This is not the case. Neither for nor while loops create a subshell.
(Score: 0) by Anonymous Coward on Tuesday December 01 2015, @09:33PM
It appears that while loops do: Bash Subshells [tldp.org]
(Score: 0) by Anonymous Coward on Tuesday December 01 2015, @09:35PM
Never mind, I just noticed that the while was enclosed in a ( ), which where the subshell comes from...
(Score: 5, Funny) by wonkey_monkey on Tuesday December 01 2015, @10:40PM
while for loops do not
Eh, so confused. I'm done.
systemd is Roko's Basilisk
(Score: 0) by Anonymous Coward on Tuesday December 01 2015, @10:42PM
tar never needed the -x or -j or -J options.
(I'm gonna assume you mean -X instead of -x since a tar that can't extract files is pretty useless!)
tar's command line is pretty bad to begin with [xkcd.com]. I prefer pax; too bad most distros don't ship it by default even though it's POSIX.
(Score: 2, Informative) by xav on Wednesday December 02 2015, @03:55AM
My favorite gotcha is the difference between the for loop and the while loop in bash. While loops create subshells with a new environment while for loops do not.
No, pipes do create subshells, not "while".
What I mean is that
# seq 1 3 | while read n; do echo $n; done
creates two subshells for seq and while, whereas
# for n in $(seq 1 3); do echo $n; done
creates only one subshell for seq, not for the "for" loop.
On the other hand, the two following while loops do the same thing in the current shell:
# while read n; do echo $n; done <<< "$(seq 1 3)"
# while read n; do echo $n; done < <(seq 1 3)
(Score: 2) by tempest on Tuesday December 01 2015, @09:04PM
My biggest CLI gocha is the quagmire of passing arguments to programs in Powershell. I keep thinking "Does it REALLY need to be this complicated?".
(Score: 0) by Anonymous Coward on Tuesday December 01 2015, @09:37PM
My biggest CLI gocha is the quagmire of Powershell. I keep thinking "Does it REALLY need to be this complicated?".
There. FTFY.
(Score: 2) by Techwolf on Tuesday December 01 2015, @10:46PM
ping 8.8.8.8 will now fail with a not so obvious error message.
One has to change all there scripts to use "ping -4" instead.
(Score: 3, Interesting) by VLM on Tuesday December 01 2015, @10:49PM
So thirty years ago the biggest problem was dropping your punch card deck when you'd submit a job; just kidding. So its hard to remember.
Fun things that still occasionally happen:
Errors in judgement with xargs in general and especially the -n option. Tends to be a performance kill not a major disaster, but its still annoying as hell. Shouldda used find and xargs there, how the heck was I to predict we'd have five hundred thousand files one day in that directory. I always kinda thought there would never be more than one LOL.
Ditto log deletion and rotation. So I turned on query logging to debug something, forgot about it for six months, come to work with a nastygram from the fileserver admins asking WTF I'm killing the whole file server by appending to the end of a nearly terabyte length log file. Deleting that helped with my quota quite a bit. Actually I think it resulted in them putting a quota on me.
Its still possible to make mistakes with tab expansion and concatenation vs redirection. vi wtf.appendtoend cat wtf.appendtoend "looks fine to me" cat wtf.appendtoend > wtf "Son of a ... I can't believe I did it again... OK... sigh .. wheres the backups..." vi /etc/passwd.demonstration to show off how badass I am, rm /etc/passwd tab tab tab hit enter, F I just nuked /etc/passwd not /etc/passwd.demonstration. Or similar. In general I get more done by slowing down and not being in a hurry. Some of my best disasters have come from being in a hurry.
(Score: 2) by darkfeline on Wednesday December 02 2015, @03:05AM
I use GNU Parallel now instead of xargs, much better in my opinion, especially not having the fudge about with whitespace handling.
Join the SDF Public Access UNIX System today!
(Score: 2) by Marand on Wednesday December 02 2015, @08:04AM
Parallel is a gotcha in and of itself, because there are two separate versions of it that have different behaviours and arguments, at least in Debian. One is part of the "moreutils" package, while the GNU one (which is awesome) is its own standalone package named parallel. You can't have both installed because they both want /usr/bin/parallel, so if you need anything else that's in moreutils, you're stuck with the shittier version of parallel.
The right solution would be to have parallel.gnu and parallel.moreutils and use Debian's alternatives system to make a symlink, or split parallel out of moreutils and let the user choose which one to keep, but it's been an issue since 2010 and as far as I know is still an ongoing source of bullshit, finger-pointing, and ego clashes even now. (It may have been corrected now, but when I last checked earlier this year it was still a problem...)
(Score: 1, Funny) by Anonymous Coward on Wednesday December 02 2015, @01:28PM
(Score: 2) by VLM on Wednesday December 02 2015, @01:44PM
I don't even use Debian anymore, mostly freebsd, but I found this
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=749355 [debian.org]
Usually these kind of things are resolved a little quicker.
(Score: 2) by Marand on Wednesday December 02 2015, @07:52PM
Yup, I was following that for a while and it had no sign of improvement up to about March of this year. It looks like progress i'd belong made finally but they're still stuck on the same problems because the gnu one deprecated the compat switch that makes it work like the other, the other doesn't care anout the gnu one, and nobody can agree on how to deal with that.
One reason this has dragged on for so long is the old maintainer, Joey Hess, got pissy over someone bringing in the technical committee after he failed to do anything with the problem for a while. He complained about it being a time problem not a technical one, objected to their involvement, and abandoned the package in a huff of 'screw you i'm taking my ball home'
(Score: 2) by darkfeline on Thursday December 03 2015, @01:32AM
That sounds like a problem with Debian and perhaps moreutils, which, in addition to having a bad maintainer, isn't very useful last I checked. Especially sponge, which can be easily done with regular shell IO redirection.
Join the SDF Public Access UNIX System today!
(Score: 2) by Marand on Thursday December 03 2015, @06:43AM
Oh, it is, definitely. It's a huge potential gotcha though, I had moreutils installed for vipe (insert text editor into a pipe chain) and went to use parallel one day and the much-inferior moreutils parallel caused me no end of trouble. I figured out the problem, went to install the gnu one, and that opened a whole new can of worms with the conflicts. I decided parallel was more useful than all of moreutils, though, so I just tossed moreutils to the bin over the whole nonsense.
Still, It's nothing but pain if you don't realise, especially if you learn about parallel from a website or person that doesn't mention there's an inferior version floating around...
(Score: -1, Flamebait) by Anonymous Coward on Wednesday December 02 2015, @12:12AM
I don't see any command line on StackOverflow, where real rockstar coders get stuff done, by begging for answers! Command lines are for old neckbeards who aren't hip!
(Score: 3, Insightful) by bziman on Wednesday December 02 2015, @03:07AM
I still find a command line a hundred times easier and more intuitive than any graphical user interface.
(Score: 2) by tibman on Wednesday December 02 2015, @04:22AM
Neal Stephenson wrote a rant/essay about that: http://www.cryptonomicon.com/beginning.html [cryptonomicon.com]
SN won't survive on lurkers alone. Write comments.
(Score: 0) by Anonymous Coward on Wednesday December 02 2015, @01:25PM
Photoshop/Krita/Call of Duty/Starcraft/Skyrim via a CLI is going to be inferior than using a GUI. So in many scenarios GUIs are better than CLIs. In theory a GUI can be a superset of a CLI therefore GUIs could be better than CLIs. But that's just theory.
(Score: 2) by TheRaven on Wednesday December 02 2015, @06:58PM
sudo mod me up
(Score: 2) by bziman on Thursday December 03 2015, @02:58AM
I find interfaces designed by people with expertise in HCI to be wholly unusable. I can't speak for anyone else. I find HCI people tend to try to speak for everyone. I don't like them.
(Score: 3, Interesting) by xav on Wednesday December 02 2015, @03:36AM
In the early 90s, Solaris made it possible to hard-link directories.
So, out of curiosity, I tried "ln -f /usr/bin /test"
Fine, it works! Now, let's delete this: "rm -r /test"
OUCH!
Now, I understand why most systems prevent users from hard-linking directories, even if it's technically possible.
(Score: -1, Troll) by Anonymous Coward on Wednesday December 02 2015, @04:06AM
Nobody should have a certain feature, just because you're a goddamn idiot who can't use it properly. How fucking selfish you are.
(Score: 3, Insightful) by Anonymous Coward on Wednesday December 02 2015, @04:14AM
A user hard linked / in one of his subdirectories, and you want to clean out his files. How fucking fucked you are.
(Score: 2) by FatPhil on Wednesday December 02 2015, @10:25AM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Informative) by TheRaven on Wednesday December 02 2015, @07:02PM
Hard links are usually disallowed to prevent cycles. Without hard linked directories, you can do a depth-first search of a filesystem and be guaranteed to terminate. With them, tools like find become a lot more complicated. Symlinks are fine, because they're distinct from normal files (hard links aren't in a traditional UNIX filesystem: there's no notion of the canonical file and the link with a hard link, both paths are equally canonical), but hard links mean that you have to track the inode for every directory that you visit and check every other one against it to prevent infinite recursion.
HFS+ does permit hard links on directories, but requires a special permission to create them. This is used by Time Machine to create hard links that point to older backups for unmodified subtrees.
sudo mod me up
(Score: 2) by FatPhil on Wednesday December 02 2015, @10:34AM
scp -P
:-p
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2) by Marand on Wednesday December 02 2015, @07:55PM
add sftp to that annoyance, too. Fuck consistency, LeT's ChAngE CAsE aT RaNdOM
(Score: 2) by PizzaRollPlinkett on Wednesday December 02 2015, @01:21PM
For some reason, I never can remember in what order the symlink parameters go. Is it "ln -s old new" or "ln -s new old"? No matter how many times I look it up, I never seem to remember.
Not a CLI gotcha, but I use scripting languages like Perl and Python so much that when I have to use bash to write a little script or something (even from the CLI), I never remember it. When I look up something, the syntax is so bizarre I remember why I use scripting languages!
(E-mail me if you want a pizza roll!)