OpenSSH 8.8 has been released and with it comes a heads up that there will be major changes to how the scp utility operates, starting in one of the next releases. Specifically, scp has been retooled to use the SFTP protocol under the hood. This will leave most behavior unchanged and most times there will be no perceived difference. However, some scripts which make use of globbing might need minor adjustment to work properly in the future:
A near-future release of OpenSSH will switch scp(1) from using the legacy scp/rcp protocol to using SFTP by default.
Legacy scp/rcp performs wildcard expansion of remote filenames (e.g. "scp host:* .") through the remote shell. This has the side effect of requiring double quoting of shell meta-characters in file names included on scp(1) command-lines, otherwise they could be interpreted as shell commands on the remote side.
This creates one area of potential incompatibility: scp(1) when using the SFTP protocol no longer requires this finicky and brittle quoting, and attempts to use it may cause transfers to fail. We consider the removal of the need for double-quoting shell characters in file names to be a benefit and do not intend to introduce bug- compatibility for legacy scp/rcp in scp(1) when using the SFTP protocol.
Another area of potential incompatibility relates to the use of remote paths relative to other user's home directories, for example - "scp host:~user/file /tmp". The SFTP protocol has no native way to expand a ~user path. However, sftp-server(8) in OpenSSH 8.7 and later support a protocol extension "expand-path@openssh.com" to support this.
The new behavior is now present in scp but currently off by default. It can be tested using the temporary -s option. Later, the -O option will force use of the original scp/rcp protocol for the cases where SFTP may be unavailable or incompatible.
Compared to scp/rcp, SFTP is a new protocol but only relatively speaking. Importantly, it has been engineered from the ground up to operate as securely as possible. In contrast, scp has been written without a formal specification other than to operate like the late rcp did, but over SSH. Currently, scp requires expansion of glob patterns using the remote system's shell. That can be eliminated by dropping scp and switching to SFTP beneath it all.
Previously:
(2019) Oh, SSH, IT Please see this: Malicious Servers can fsck with your PC's Files During scp Slurps
(2018) OpenSSH SFTP Chroot Code Execution
(2014) OpenSSH No Longer has to Depend on OpenSSL
Related Stories
What has been planned for a long time now, prior to the infamous heartbleed fiasco of OpenSSL (which does not affect SSH at all), is now officially a reality - with the help of some recently adopted crypto from DJ Bernstein. OpenSSH now finally has a compile-time option to no longer depend on OpenSSL, the option `make OPENSSL=no` has now been introduced for a reduced-configuration OpenSSH to be built without OpenSSL.
The result would leave you with no legacy SSH-1 baggage at all, and on the SSH-2 front with only AES-CTR and chacha20+poly1305 ciphers, ECDH/curve25519 key exchange and Ed25519 public keys.
[Editor's Note: This appears to be very much a Work-in-Progress, so might not be available for your distro or via standard repositories.]
The SFTP component in OpenSSH provides a chroot-feature for hardening. It is stated in the documentation that the chroot directory must not be writable by the user account, though specific files and subdirectories within it are allowed. Some people were questioning the read-only restriction. halfdog documents some analysis which is the result of discussions on openssh-dev mailing list. Here are some arguments about why these restrictions still makes sense in 2018.
Oh, SSH, IT Please see This: Malicious Servers can Fsck With Your PC's Files During scp Slurps
A decades-old oversight in the design of Secure Copy Protocol (SCP) tools can be exploited by malicious servers to unexpectedly alter victims' files on their client machines, it has emerged.
F-Secure's Harry Sintonen discovered a set of five CVE-listed vulnerabilities, which can be abused by evil servers to overwrite arbitrary files on a computer connected via SCP. If you use a vulnerable version of OpenSSH's scp, PuTTY's PSCP, or WinSCP, to securely transfer files from a remote server, that server may be able to secretly tamper with files on your local box that you do not expect the server to change.
[...] Sintonen explained that because rcp, on which scp is based, allows a server to control which files are sent, and without the scp client thoroughly checking it's getting its expected objects, an attacker can do things like overwrite the user's .bash_aliases file. This, in turn, would allow the attacker to run arbitrary commands on the victim's box when the user does routine stuff, like list a directory.
"Many scp clients fail to verify if the objects returned by the scp server match those it asked for. This issue dates back to 1983 and rcp, on which scp is based," Sintonen explained in his disclosure this month.
"A separate flaw in the client allows the target directory attributes to be changed arbitrarily. Finally, two vulnerabilities in clients may allow server to spoof the client output."
The CVE (Common Vulnerabilities and Exposures) reports are:
- CVE-2018-20685
- CVE-2019-6111
- CVE-2018-20684
- CVE-2019-6109
- CVE-2019-6110
Only WinSCP seems to have released an update that fixes these.
OpenSSH developer, Damien Miller, has announced plans to remove support for DSA keys from OpenSSH in the near future. His announcement describes the rationale, process, and proposed timeline.
The next release of OpenSSH (due around 2024/03) will make DSA optional at compile time, but still enable it by default. Users and downstream distributors of OpenSSH may use this option to explore the impact of DSA removal in their environments, or to hard-deprecate it early if they desire.
Around 2024/06, a release of OpenSSH will change this compile-time default to disable DSA. It may still be enabled by users/distributors if needed.
Finally, in the first OpenSSH release after 2025/01/01 the DSA code will be removed entirely.
In summary:
2024/01 - this announcement
2024/03 (estimated) - DSA compile-time optional, enabled by default
2024/06 (estimated) - DSA compile-time optional, *disabled* by default
2025/01 (estimated) - DSA is removed from OpenSSH
Very few will notice this change. However, for those few to whom this matters the effects are major.
Previously:
(2021) scp Will Be Replaced With sftp Soon
(2020) SHA-1 to be Disabled in OpenSSH and libssh
(2019) How SSH Key Shielding Works
(2016) Upgrade Your SSH Keys
(2014) OpenSSH No Longer has to Depend on OpenSSL
(Score: 0, Offtopic) by c0lo on Tuesday September 28 2021, @08:20AM (24 children)
What is to comment about it?
I can't even pretend I have something to be nostalgic in re scp, never had any adventure with it, not even a one-night stand or a pub crawling on its account.
Q: should we treat them differently, together with vulnerabilities/security announcements? Like this one [soylentnews.org], does it worth keeping the "story time slot" for its entirety? Its a piece of info, you read it then either "Ok, I need to do something about it" (unlikely) or "Meh... et alors?" (a lot likely)
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 5, Interesting) by maxwell demon on Tuesday September 28 2021, @08:41AM (3 children)
You could for example discuss whether you agree or disagree with the design decision. After all, breaking backwards compatibility is not to be taken lightly, and it surely would not have been hard to simply detect and remove extra quotes from file names, which should fix the majority of compatibility issues.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 3, Interesting) by canopic jug on Tuesday September 28 2021, @09:07AM
Indeed. I gather that another option would have been just to pull the plug on scp(1) and let people deal with the fallout. That would have been interesting, to say the least but was probably never a serious option even if discussed. It would have had my vote though.
This looks like it might be a viable compromise keeping most of the interface the same while removing just the legacy protocol which was causing the problems. It's not something that has been done in haste. I recall people discussing the future demise of scp(1) more than 10 years ago and recommending a move to SFTP even then, though not specifically like this use of SFTP.
Money is not free speech. Elections should not be auctions.
(Score: 2, Interesting) by c0lo on Tuesday September 28 2021, @11:29AM
Honest to God, cross-your-heart... if I would not have posted my comment, would the topic of your example popped into your mind as worthy to comment on?
(I have this hunch you wouldn't, because you didn't consider important enough to open a special thread for it. It resembles more of a "Now that you mentioned it, how about..."
Of course I can't be sure and of course I'm not judgemental no matter your answer)
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by FatPhil on Tuesday September 28 2021, @06:33PM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 3, Insightful) by canopic jug on Tuesday September 28 2021, @08:51AM (9 children)
There's nothing nostalgic about scp(1). Good riddance. scp(1) has needed to go for a while so that system updates can propagate the needed changes though the world. The scp(1) problem has had a few decades to spread and become pervasive and passive clean up has not done the job. scp(1) was the go-to method for file transfer for a long time and all too many guides and tutorials, new and old, still persist in mentioning it instead of Rsync or SFTP. And even young people are thus prompted to turn to the less-than-optimal solution of scp(1) in their new scripts. Once the habit takes, it is hard to change. Now push has come to shove and the only way turns out to be to leave the user interface in place and swap out the underlying protocol.
The reason I wrote the post is that most remotely managed systems use SSH, usually OpenSSH at that. I'd say therefore that people here probably use SSH frequently in one capacity or another. Many may even have scripts, either their own scripts or ones that they have inherited. Some of the scripts may have scp(1) hidden in there instead of using SFTP or Rsync. Granted the various distros will take a long time to push out just these changes, especially with the long term support editions. But the changes affect two common annotations, so this is the kind of change that needs a lead time to deal with properly and all new file transfer scripts need to work with the new scp(1) or, better, directly with SFTP or Rsync.
Money is not free speech. Elections should not be auctions.
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @09:04AM (4 children)
I feel that there is some sort of message in your spelling of "scp" as "scp(1)", but I don't get it.
If it's an important message, could you please phrase it more clearly?
(Score: 2) by canopic jug on Tuesday September 28 2021, @09:10AM
No message, it is just the relevant section of the manual pages. It's the annotation that was used in the announcement but got omitted from the summary.
Money is not free speech. Elections should not be auctions.
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @09:16AM
It's meaningless unless you use *NIX at a terminal. Just a shorthand for those who might be reading the manuals.
(Score: 3, Informative) by Anonymous Coward on Tuesday September 28 2021, @09:47AM (1 child)
The "name(number)" is used in man pages to distinguish the man page called "name" in the section with that number from other pages called "name" that appear in other sections. In the case of "scp(1)" it is being used to specifically refer to the item that corresponds with the page called "scp" in section 1 "general commands." It is a quick way and unambiguous way to signify that you are talking about a program called "scp" and not the protocol called "scp" which would be denoted with "scp(7)" or "scp(5)" depending on the section system used.
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @10:43AM
Thank you.
(Score: 2) by maxwell demon on Tuesday September 28 2021, @11:10AM
Of course what is replaced is not the tool, but the protocol. The tool still exists, it just uses the sftp protocol underneath, and has a few incompatibilities to the current scp.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by c0lo on Tuesday September 28 2021, @11:19AM
Apologies, that was a tongue-in-cheek on which I forgot to (grin).
Which makes from it a "community announcement" - beware guys there are some changes that may bite your ass.
The best comment I can post of this kind of post is "Thank you for the tip" (if it helps me). I don't think that a whole page of "Thank you" comments worth the entire time slot reserved for a story.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @06:50PM (1 child)
scp is in muscle memory. And, it will continue to be after this back-end switch-- it is simple and handy for simple file transfers. For more complex transfers, I've always just run remote commands (e.g., tar, dump/restore, etc.) via ssh (not an interactive shell) and piped output/input. I think this is fairly common. And, folks doing this to handle the more complex stuff won't notice anything, from this change, either.
The way the openbsd team are handling this is great, though. For most folks, the change probably won't be noticed. If you are doing something tricky with scp you might get bit by this, but also nice how they are announcing this way ahead of time, and hiding old behavior behind a switch as an emergency escape for, "oh f@%$, my script is broke, how do I fix it *now*?". And, having a switch ahead of time to pre-test your scripts is even better.
They have been warning us for many years that this was their eventual plan. At work, most scripts that used scp were re-written to use sftp both in preparation for deprecation (10 yrs ?or more? they have been warning of deprecation of scp) and, also, so we can chroot the users easily.
There is one script that uses undocumented behavior of scp when combined with a forced command to override dest path+file that will need to be updated still. No way that I know of of forcing sftp (server-side) to only write to a single file at a single path regardless of what the client side specifies. I guess sftp into a chroot will have to suffice.
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @09:21PM
I tend to use the -e ssh option of rsync in most cases for those kind of processes, but, you can always tunnel other sorts of things between computers.
(Score: 4, Interesting) by janrinok on Tuesday September 28 2021, @10:34AM (2 children)
If you are not a programmer, administrator, or a frequent user of the command line it will probably not affect how you use your computer.
However, for those working in such roles this is an important issue. It is quite possible, as Canopic Jug has pointed out, that scripts that are in frequent use might suddenly stop working the way that we expect them to. I don't know what proportion of our community this story will affect currently but in the past it was certainly a significant chunk - so much so that we actually have the Sortware topic for such stories.
But not all stories interest everyone in our community. We have stretched the definitions to allow a relatively wide span of topics to be discussed, but this is precisely the sort of discussion that many enjoyed in the early days of the site. It is informative and, for some, might be an important topic if scp is used in their daily professional work where this might actually result in a company experiencing an unexpected problem.
(Score: 1, Offtopic) by c0lo on Tuesday September 28 2021, @11:11AM
I'm making quite a good living software-engineering new things (not jusr programming them). Not that doing so necessary require the use of ssh or scp, it has been quite a long time since I needed it. You see, the use of a SCM (e.g. git) is much more prevalent in the category of "File operations" and automatic deploy are more frequently done with specific tools.
I'm not saying those kind of stories don't have a good role. But I don't expect extensive comments on such a story, it's mostly "informational" in nature (this is why my use of "public community announcement").
I didn't suggest to eliminate them completely, just don't let them live alone on the top for a whole 3 hours or whatever the cadence of publishing new stories, I don't think they worth it.
Well, times change. The latest ones that I'm seeing, barely make 5 comments and it's not unusual to find a couple of "Frost piss" vandalism among those comments.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 4, Informative) by fliptop on Tuesday September 28 2021, @11:48AM
I use scp every day, many times per day, to move files around between machines. I've been doing sysadmin since the late 90's and a lot of my legacy scripts use it for copying db dumps to the backup server. I use rsync too for static files and mail backups, but I like having a separate cron job for copying db dumps so the report of what happened doesn't get buried in a mile-long rsync report.
To me, stories like this are much appreciated.
Our Constitution was made only for a moral and religious people. It is wholly inadequate to the government of any other.
(Score: 5, Interesting) by PiMuNu on Tuesday September 28 2021, @10:58AM (4 children)
Quite often I read these sorts of stories and think "thanks, that was really useful, but I don't have a comment". So I see interesting stories going past with low comment tallies and hope that the eds don't think the story is worthless just because of the low tally. Makes me wish that I could mod against a story (but clearly such a system is open to abuse)
(Score: 1, Offtopic) by c0lo on Tuesday September 28 2021, @11:14AM (2 children)
My suggestion was: post them, don't wait to get community participation.
And most of the "charm" of S/N stay rather in commenting then on the stories themselves - letting them linger the whole time until the next scheduled (commenting favorable) FA is a sorta waste.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 3, Interesting) by PiMuNu on Tuesday September 28 2021, @11:46AM (1 child)
> most of the "charm" of S/N stay rather in commenting then on the stories themselves
I agree, but there is some "charm" in weird little technical details (much like this one). Comments alone would lean the eds towards always posting another AGW/covid/etc story. That's fine once in a while, but it tends to feed the trolls too much. Maybe the eds already take that into account.
(Score: 2) by c0lo on Tuesday September 28 2021, @12:10PM
Show that you like it. I can form no idea if you say nothing, not even a "I like this story, thanks for it".
I liked this one [soylentnews.org] well enough to try to share it with whoever from soylenters would find it interesting. I have no idea if I should continue to do it or should I keep the delight for myself. Perhaps that single on-topic reaction may suggest I should continue posting from time to time such stories.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @04:08PM
I have nothing meaningful to say about both the story and the comment, but I agree with the comment, so am posting just to note that (and add one more comment to the "popularity content" of this story).
(Score: 1, Interesting) by Anonymous Coward on Wednesday September 29 2021, @07:41PM
For me, "zero days under active exploit, exponential growth in detections" is news. So is "compatability-breaking changes in a cli tool incoming."
In the physical world, my news stream includes local disaster news.
I live in a digital world, where local disaster news is somewhat broader. Yes, this is neighbourhood news - pileup on the old wooden bridge might force interstate bypasses lengthening trips by 20min! gasp! - but for the few people in those cars or on the way to critically timed jobs, it's news, and those of us on these forums are grepping our home and work scripts for scp, instead of checking names on a chyron for friends and relatives.
(Score: 3, Insightful) by DECbot on Tuesday October 05 2021, @11:38PM
I know I'm late to the discussion, but this is one of those stories that I appreciate but expect little discussion. Though today I'm delightfully surprised to see so much of it.
scp has stuck with me over the years despite its impending deprecation because the syntax is so like cp but with the relevant ssh bits tacked on to make the cp command transcend the system you're logged into. The syntax is obvious and I've never really needed to reference man pages or google the command to make sure I'm using the flags correctly for moving a single file or folder across the network. The article is a good reminder that I should spend more time familiarizing myself to alternatives like sftp and rsync. It is also interesting that some distros are changing the underlying functionality of the utility as opposed to removing it from the repo. This is good info to have when you're moving files from a 2012 circa server to a 2022 circa server and they're just not agree on how that is accomplished. As it's been said many times elsewhere, you'll see this a bunch in older bash scripts that still expect the vulnerable scp functionality. Instead of banging your head on the wall going "wtf! why is this not working?" you go "oh yeah, that fking changed," change the utility, and move on without wasting time troubleshooting why a deprecated utility isn't working.
cats~$ sudo chown -R us /home/base
(Score: -1, Offtopic) by Anonymous Coward on Tuesday September 28 2021, @09:11AM
Curse my metal body. I wasn't fast enough. It's all my fault. My poor master.
(Score: 5, Informative) by digitalaudiorock on Tuesday September 28 2021, @03:37PM (2 children)
Somehow way back I got in the habit of using rsync instead of scp. Whenever I bring that up it seems like many dislike it for some reason, and find it confusing, but I've always preferred it. Having said that, it's worth noting that it's not always an option, as it requires that the remote side actually has rsync installed, whereas sftp and scp just use existing protocols.
I'd say that a few of the important things to understand about using rsync, the first being the most important, is that a references to source directories with a trailing slash like this:
...will pull the contents of that directory, whereas this:
...will get the directory itself. The same goes for references to local directories that you might by sending elsewhere. That's especially important to know because tab completion on directories tends to add that trailing slash...so it's an easy mistake to make.
Another good tip is how to reference multiple remote directories on the same host. You can do that like this example:
That is ":path" assumes the previous user/server.
The other advantage or rsync is of course for actually ryncing directories, which is capable of sending only deltas etc. Like I said, this has always been my go-to program for remote transfers.
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @06:29PM (1 child)
I've been bitten by the trailing slash thing before and wonder why rsync operates this way as opposed to most other unix-y things. ie. why not require a wildcard like
to copy directory contents? Too late to change now without breaking existing scripts I guess...
(Score: 2) by digitalaudiorock on Wednesday September 29 2021, @09:34AM
One issue with doing that is that the concept needs to apply to local directories as well, so then you'd have the issue that the shell would expand wildcards when you don't want it to. That would turn into a mess very quickly I think.
(Score: 3, Interesting) by epitaxial on Tuesday September 28 2021, @04:22PM (2 children)
Every time I've used these protocols it seems like they never transferred at anything close to the interface speed, especially over my broadband connection. I was getting like 350kb/s over a 100mbit connection.
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @05:39PM
speed in limited by complexity (multiple processes passing handshakes (best case is four processes on two machines all in sync)) compounded by latency (all four process touch every handshake with both inter-process and network delays) complicated by cryptography (not only are the sensitive (and not so sensitive) files encrypted so are the handshakes). For better speed archive the files (so you are only transmitting large files) and increase the transmit window (if you can).
(Score: 1, Informative) by Anonymous Coward on Tuesday September 28 2021, @07:01PM
With hpc-ssh, you can get as close to wire speed as you can with any other protocol. In fact, faster than many with their changes to ack handling (depending on network topology-- greatest speedup from ack handling is if you are transferring over high latency links, e.g., between continents / over satellite links).
The null encryption option they add, still uses encryption for the authentication phase, it is just the bulk data transfer that is unencrypted-- very useful if speed is what you crave / you are already transferring inside an encrypted vpn tunnel. You can take advantage of the other patches and still use encrypted transfers too.
https://www.psc.edu/hpn-ssh-home/ [psc.edu]
https://github.com/rapier1/openssh-portable [github.com]
From a packet capture some years ago, I'm pretty sure Veeam is using hpc-ssh for all its data transfers during backup / restore.