Written Jan 15 2019.
Recommendation: sftp for copying files between machines.
sftp
is a better tool than scp
for copying files between machines. It doesn’t (mis)interpret filenames and has a more secure design overall. For example it has a built in chroot system.
Disadvantages of sftp: “SCP is usually much faster than SFTP at transferring files, especially on high latency networks”. From [https://superuser.com/questions/134901/whats-the-difference-between-scp-and-sftp]. I timed a copy across my local network of a 104MB file - sftp: 37 seconds. scp: 34 seconds.
A discussion about scp vs sftp from curl dev. [https://ec.haxx.se/usingcurl-scpsftp.html]
Why shouldn’t I use scp?
It interprets the filename as a shell command on the remote host before trying to copy that file. e.g. scp pi:'`touch wow`' .
creates a file on the ‘pi’ computer.
Some people have called this a “feature” rather than a bug and discussed how you can do things like scp pi:/db/backup-$(date +%s)
. I don’t really buy this, do you know anybody that makes use of this feature on purpose?
So you have to do a bit of awkward/difficult shell sorcery with "
and '
to copy filenames with spaces or sigils. It can be complex when there are 4 different possible levels of quoting and interpretation in a single command (local shell processing, command line options processing, parsing of inputs, remote shell processing).
Also there was some interesting research finding even more nasty bugs with scp: [https://blog.f-secure.com/weak-scp-security-vulnerabilities/]. (Security note: This is very minor in terms of security, it can only be exploited if you ssh into a malicious host. For that to happen you would have to accept a bad fingerprint. On the other hand it illustrates the bad design of scp). It’s such an old protocol that it’s really outdated in the way it’s designed. They are sort of forced to keep the unpleasant features in because changing anything would break millions of scripts.
What about rsync
rsync might have some nice theory to the transfer algorithm: [https://rsync.samba.org/tech_report/] but the actual implementation leaves a lot to be desired. It’s got a long long history of quite serious security vulnerabilities involving pathnames. I don’t recommend using this tool at all.
How do I use sftp
copy a file from my local computer to my remote computer
echo put 'sheep.txt' | sftp pi
echo put 'sheep.txt' here\\\'s-sheep.txt | sftp pi
echo put 'sheep.txt' "'heres sheep.txt'" | sftp pi
Note: You do need to escape the filenames in your ‘put’ commands.
copy a file from my remote computer to my local computer
sftp pi:file-i-want.txt
or
sftp pi:file-i-want.txt mine-now.txt
note: this will go into interactive mode if the filename actualy designates a directory on the remote.
copy a folder from my local computer to my remote computer
echo get -r "'the folder'" | sftp pi
or
echo get -r "'your folder'" "'my folder'" | sftp pi
Note: does not follow symbolic links
copy a folder from my remote computer to my local computer
echo put -r site-backup/ | sftp pi
Note: does not follow symbolic links
Tab completion of remote filenames?
Yes! You can get tab completion. Just make sure to use pubkey auth not password auth.
what commands does sftp let me use?
sftp> help
Available commands:
bye Quit sftp
cd path Change remote directory to 'path'
chgrp grp path Change group of file 'path' to 'grp'
chmod mode path Change permissions of file 'path' to 'mode'
chown own path Change owner of file 'path' to 'own'
df [-hi] [path] Display statistics for current directory or
filesystem containing 'path'
exit Quit sftp
get [-afPpRr] remote [local] Download file
reget [-fPpRr] remote [local] Resume download file
reput [-fPpRr] [local] remote Resume upload file
help Display this help text
lcd path Change local directory to 'path'
lls [ls-options [path]] Display local directory listing
lmkdir path Create local directory
ln [-s] oldpath newpath Link remote file (-s for symlink)
lpwd Print local working directory
ls [-1afhlnrSt] [path] Display remote directory listing
lumask umask Set local umask to 'umask'
mkdir path Create remote directory
progress Toggle display of progress meter
put [-afPpRr] local [remote] Upload file
pwd Display remote working directory
quit Quit sftp
rename oldpath newpath Rename remote file
rm path Delete remote file
rmdir path Remove remote directory
symlink oldpath newpath Symlink remote file
version Show SFTP version
!command Execute 'command' in local shell
! Escape to local shell
? Synonym for help
Alternatively
tar cf - . | ssh host 'cd path && tar xf -'
cheers Taylor.