Secure File Copy (scp) Tool

Thursday, 3 March 2022

Secure File Copy (scp) Tool

scp is used to securely copy files and directories to or from remote machine.

$man scp

SCP(1)                                                                                 BSD General Commands Manual                                                                                SCP(1)

NAME
     scp — OpenSSH secure file copy

SYNOPSIS
     scp [-346BCpqrTv] [-c cipher] [-F ssh_config] [-i identity_file] [-J destination] [-l limit] [-o ssh_option] [-P port] [-S program] source ... target

DESCRIPTION
     scp copies files between hosts on a network. It uses ssh(1) for data transfer, and uses the same authentication and provides the same security as ssh(1). scp will ask for passwords or
     passphrases if they are needed for authentication.

     The source and target may be specified as a local pathname, a remote host with optional path in the form [user@]host:[path], or a URI in the form scp://[user@]host[:port][/path]. Local file
     names can be made explicit using absolute or relative pathnames to avoid scp treating file names containing ‘:’ as host specifiers.

     When copying between two remote hosts, if the URI format is used, a port may only be specified on the target if the -3 option is used.

     The options are as follows:

     -3      Copies between two remote hosts are transferred through the local host. Without this option the data is copied directly between the two remote hosts. Note that this option disables the
             progress meter.

     -4      Forces scp to use IPv4 addresses only.

     -6      Forces scp to use IPv6 addresses only.

     -B      Selects batch mode (prevents asking for passwords or passphrases).

     -C      Compression enable. Passes the -C flag to ssh(1) to enable compression.

     -c cipher
             Selects the cipher to use for encrypting the data transfer. This option is directly passed to ssh(1).

     -F ssh_config
             Specifies an alternative per-user configuration file for ssh. This option is directly passed to ssh(1).

     -i identity_file
             Selects the file from which the identity (private key) for public key authentication is read. This option is directly passed to ssh(1).

     -J destination
             Connect to the target host by first making an scp connection to the jump host described by destination and then establishing a TCP forwarding to the ultimate destination from there. Mul‐
             tiple jump hops may be specified separated by comma characters. This is a shortcut to specify a ProxyJump configuration directive. This option is directly passed to ssh(1).

     -l limit
             Limits the used bandwidth, specified in Kbit/s.

     -o ssh_option
             Can be used to pass options to ssh in the format used in ssh_config(5). This is useful for specifying options for which there is no separate scp command-line flag. For full details of
             the options listed below, and their possible values, see ssh_config(5).

                   AddressFamily
                   BatchMode
                   BindAddress
                   BindInterface
                   CanonicalDomains
                   CanonicalizeFallbackLocal
                   CanonicalizeHostname
                   CanonicalizeMaxDots
                   CanonicalizePermittedCNAMEs
                   CASignatureAlgorithms
                   CertificateFile
                   ChallengeResponseAuthentication
                   CheckHostIP
                   Ciphers
                   Compression
                   ConnectionAttempts
                   ConnectTimeout
                   ControlMaster
                   ControlPath
                   ControlPersist
                   GlobalKnownHostsFile
                   GSSAPIAuthentication
                   GSSAPIDelegateCredentials
                   HashKnownHosts
                   Host
                   HostbasedAuthentication
                   HostbasedKeyTypes
                   HostKeyAlgorithms
                   HostKeyAlias
                   Hostname
                   IdentitiesOnly
                   IdentityAgent
                   IdentityFile
                   IPQoS
                   KbdInteractiveAuthentication
                   KbdInteractiveDevices
                   KexAlgorithms
                   LogLevel
                   MACs
                   NoHostAuthenticationForLocalhost
                   NumberOfPasswordPrompts
                   PasswordAuthentication
                   PKCS11Provider
                   Port
                   PreferredAuthentications
                   ProxyCommand
                   ProxyJump
                   PubkeyAcceptedKeyTypes
                   PubkeyAuthentication
                   RekeyLimit
                   SendEnv
                   ServerAliveInterval
                   ServerAliveCountMax
                   SetEnv
                   StrictHostKeyChecking
                   TCPKeepAlive
                   UpdateHostKeys
                   User
                   UserKnownHostsFile
                   VerifyHostKeyDNS

     -P port
             Specifies the port to connect to on the remote host. Note that this option is written with a capital ‘P’, because -p is already reserved for preserving the times and modes of the file.

     -p      Preserves modification times, access times, and modes from the original file.

     -q      Quiet mode: disables the progress meter as well as warning and diagnostic messages from ssh(1).

     -r      Recursively copy entire directories. Note that scp follows symbolic links encountered in the tree traversal.

     -S program
             Name of program to use for the encrypted connection. The program must understand ssh(1) options.

     -T      Disable strict filename checking. By default when copying files from a remote host to a local directory scp checks that the received filenames match those requested on the command-line
             to prevent the remote end from sending unexpected or unwanted files. Because of differences in how various operating systems and shells interpret filename wildcards, these checks may
             cause wanted files to be rejected. This option disables these checks at the expense of fully trusting that the server will not send unexpected filenames.

     -v      Verbose mode. Causes scp and ssh(1) to print debugging messages about their progress. This is helpful in debugging connection, authentication, and configuration problems.

EXIT STATUS
     The scp utility exits 0 on success, and >0 if an error occurs.

SEE ALSO
     sftp(1), ssh(1), ssh-add(1), ssh-agent(1), ssh-keygen(1), ssh_config(5), sshd(8)

HISTORY
     scp is based on the rcp program in BSD source code from the Regents of the University of California.

AUTHORS
     Timo Rinne <tri@iki.fi>
     Tatu Ylonen <ylo@cs.hut.fi>

BSD                                                                                         November 30, 2019

To copy a directory recursively use -r flag:

$ scp -r nvidia@nvidia-nano:/usr/src/tensorrt/samples/python/uff_ssd ~/dev/
nvidia@nvidia-nano's password:
inference.py 100% 12KB 3.3MB/s 00:00
engine.py 100% 5750 4.5MB/s 00:00
__init__.py 100% 0 0.0KB/s 00:00
boxes.py 100% 6725 5.2MB/s 00:00
coco.py 100% 4755 4.2MB/s 00:00
...

shell - How do I copy a folder from remote to local using scp? - Stack Overflow

To copy file which has spaces in path and/or name use double backslash before space and wrap entire file path in double quotation marks:

~/Downloads$ scp nvidia@nvidia-nano:"/home/nvidia/Pictures/Object\\ Detection\\ -\\ SSD.png" ~/Pictures
nvidia@192.168.0.10's password:
Object Detection - SSD.png 100% 444KB 1.2MB/s 00:00

To copy a directory from local to remote:

$ scp -r LocalDir user@19.168.0.61:/home/user/destination

This will create LocalDir in destination folder.

If public-private key (SSH keys) authentication is used (instead of username/password) we can specify the path to the private key. For the following example we've created key pair for accessing AWS EC2 VM instance, downloaded the private key (.pem) file and can use it to access that VM in order to copy a directory onto it:

~/.ssh$ scp \

-i "my-vm.pem" \

-pr /home/bojan/dev/my-app/ \

ec2-user@50.51.52.53:/home/ec2-user/my-app/

This is how to copy a file from remote (EC2 instance in this case) to a local host (into a current directory):

$ scp \

-i key-pair--ec2--bojan-temp.pem \

ec2-user@ec2-19-208-98-120.compute-1.amazonaws.com:/usr/bin/my_file \

To copy a single file from local to remote:

$ scp \

-i key-pair--ec2--bojan-temp.pem \

my_file \

ec2-user@ec2-19-208-99-120.compute-1.amazonaws.com:/home/ec2-user/

To copy all files from current directory, we can use *:

~/path/to/my-app$ scp \

-i ~/.ssh/my-vm.pem \

-pr * \

ec2-user@50.51.52.53:/home/ec2-user/my-app/

app.py                                                                                                                                                                      100% 4921    17.9KB/s   00:00
Dockerfile                                                                                                                                                                  100% 152     0.9KB/s   00:00
Makefile                                                                                                                                                                    100% 313     2.1KB/s   00:00
README.md                                                                                                                                                                   100% 2179     8.1KB/s   00:00
requirements.txt

scp preserves file/directory attributes e.g. if it was hidden on the origin machine, it will also be hidden on the target machine.

Troubleshooting

If SSH server is not installed, not running or firewall blocks incoming connections on port 22 on the remote machine, scp command might fail with this error:

$ scp -r local_dir/ user@remote_host:~/Downloads/
ssh: connect to host remote_host port 22: Connection refused
lost connection

To fix this, we need to have access to the remote machine so we can check if the SSH server is installed and is up and running.

The following commands show the case when openssh server is not installed:

To check if sshd process is running:

$ ps -aux | grep sshd

To check if there is SSH server config file (which existence is mandatory for running SSH server):

$ cat /etc/ssh/sshd_config
cat: /etc/ssh/sshd_config: No such file or directory

To check which openssh application is installed (the output shows that only SSH client is present):

$ apt list --installed | grep openssh

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

openssh-client/jammy-updates,now 1:8.9p1-3ubuntu0.1 amd64 [installed,automatic]

To install SSH server:

$ sudo apt install openssh-server

We can now verify again all installed openssh packages:

$ apt list --installed | grep openssh

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

openssh-client/jammy-updates,now 1:8.9p1-3ubuntu0.1 amd64 [installed,automatic]
openssh-server/jammy-updates,now 1:8.9p1-3ubuntu0.1 amd64 [installed]
openssh-sftp-server/jammy-updates,now 1:8.9p1-3ubuntu0.1 amd64 [installed,automatic]

SSH server config is now present:

$ cat /etc/ssh/sshd_config

# This is the sshd server system-wide configuration file. See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented. Uncommented options override the
# default value.

Include /etc/ssh/sshd_config.d/*.conf

#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#PubkeyAuthentication yes

# Expect .ssh/authorized_keys2 to be disregarded by default in future.
#AuthorizedKeysFile    .ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
KbdInteractiveAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the KbdInteractiveAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via KbdInteractiveAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and KbdInteractiveAuthentication to 'no'.
UsePAM yes

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

# override default of no subsystems
Subsystem    sftp    /usr/lib/openssh/sftp-server

# Example of overriding settings on a per-user basis
#Match User anoncvs
#    X11Forwarding no
#    AllowTcpForwarding no
#    PermitTTY no
#    ForceCommand cvs server

Installing openssh-server is enough. If we re-run now scp command, it will be successful.

We can also verify that ssh daemon is listening on port 22:

$ sudo lsof -i :22

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

sshd 1737 root 3u IPv4 30631 0t0 TCP *:ssh (LISTEN)

sshd 1737 root 4u IPv6 30633 0t0 TCP *:ssh (LISTEN)