SoylentNews is people
SoylentNews
OpenSSH 8.8 has been released and with it comes a heads up that there will be major changes to how the scp utility operates, starting in one of the next releases. Specifically, scp has been retooled to use the SFTP protocol under the hood. This will leave most behavior unchanged and most times there will be no perceived difference. However, some scripts which make use of globbing might need minor adjustment to work properly in the future:
A near-future release of OpenSSH will switch scp(1) from using the legacy scp/rcp protocol to using SFTP by default.
Legacy scp/rcp performs wildcard expansion of remote filenames (e.g. "scp host:* .") through the remote shell. This has the side effect of requiring double quoting of shell meta-characters in file names included on scp(1) command-lines, otherwise they could be interpreted as shell commands on the remote side.
This creates one area of potential incompatibility: scp(1) when using the SFTP protocol no longer requires this finicky and brittle quoting, and attempts to use it may cause transfers to fail. We consider the removal of the need for double-quoting shell characters in file names to be a benefit and do not intend to introduce bug- compatibility for legacy scp/rcp in scp(1) when using the SFTP protocol.
Another area of potential incompatibility relates to the use of remote paths relative to other user's home directories, for example - "scp host:~user/file /tmp". The SFTP protocol has no native way to expand a ~user path. However, sftp-server(8) in OpenSSH 8.7 and later support a protocol extension "expand-path@openssh.com" to support this.
The new behavior is now present in scp but currently off by default. It can be tested using the temporary -s option. Later, the -O option will force use of the original scp/rcp protocol for the cases where SFTP may be unavailable or incompatible.
Compared to scp/rcp, SFTP is a new protocol but only relatively speaking. Importantly, it has been engineered from the ground up to operate as securely as possible. In contrast, scp has been written without a formal specification other than to operate like the late rcp did, but over SSH. Currently, scp requires expansion of glob patterns using the remote system's shell. That can be eliminated by dropping scp and switching to SFTP beneath it all.
Previously:
(2019) Oh, SSH, IT Please see this: Malicious Servers can fsck with your PC's Files During scp Slurps
(2018) OpenSSH SFTP Chroot Code Execution
(2014) OpenSSH No Longer has to Depend on OpenSSL
(Score: 5, Informative) by digitalaudiorock on Tuesday September 28 2021, @03:37PM (2 children)
Somehow way back I got in the habit of using rsync instead of scp. Whenever I bring that up it seems like many dislike it for some reason, and find it confusing, but I've always preferred it. Having said that, it's worth noting that it's not always an option, as it requires that the remote side actually has rsync installed, whereas sftp and scp just use existing protocols.
I'd say that a few of the important things to understand about using rsync, the first being the most important, is that a references to source directories with a trailing slash like this:
...will pull the contents of that directory, whereas this:
...will get the directory itself. The same goes for references to local directories that you might by sending elsewhere. That's especially important to know because tab completion on directories tends to add that trailing slash...so it's an easy mistake to make.
Another good tip is how to reference multiple remote directories on the same host. You can do that like this example:
That is ":path" assumes the previous user/server.
The other advantage or rsync is of course for actually ryncing directories, which is capable of sending only deltas etc. Like I said, this has always been my go-to program for remote transfers.
(Score: 0) by Anonymous Coward on Tuesday September 28 2021, @06:29PM (1 child)
I've been bitten by the trailing slash thing before and wonder why rsync operates this way as opposed to most other unix-y things. ie. why not require a wildcard like
to copy directory contents? Too late to change now without breaking existing scripts I guess...
(Score: 2) by digitalaudiorock on Wednesday September 29 2021, @09:34AM
One issue with doing that is that the concept needs to apply to local directories as well, so then you'd have the issue that the shell would expand wildcards when you don't want it to. That would turn into a mess very quickly I think.