By:
Bulat Khamitov
We hope you find this tutorial helpful. In addition to guides like this one, we provide simple cloud infrastructure for developers. Learn more →

How To Copy Files With Rsync Over SSH

PostedMarch 29, 2013 588.4k views Linux Basics

Step 1 - Setup public SSH keys

On our origin server, we will generate public SSH keys with no password:

ssh-keygen -f ~/.ssh/id_rsa -q -P ""
cat ~/.ssh/id_rsa.pub

This is our public SSH key that can be placed on other hosts to give us access:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLVDBIpdpfePg/a6h8au1HTKPPrg8wuTrjdh0QFVPpTI4KHctf6/FGg1NOgM++hrDlbrDVStKn/b3Mu65//tuvY5SG9sR4vrINCSQF++a+YRTGU6Sn4ltKpyj3usHERvBndtFXoDxsYKRCtPfgm1BGTBpoSl2A7lrwnmVSg+u11FOa1xSZ393aaBFDSeX8GlJf1SojWYIAbE25Xe3z5L232vZ5acC2PJkvKctzvUttJCP91gbNe5FSwDolE44diYbNYqEtvq2Jt8x45YzgFSVKf6ffnPwnUDwhtvc2f317TKx9l2Eq4aWqXTOMiPFA5ZRM/CF0IJCqeXG6s+qVfRjB root@cloudads

Copy this key to your clipboard and login to your destination server.

Place this SSH key into your ~/.ssh/authorized_keys file:

If your SSH folder does not exist, create it manually:

mkdir ~/.ssh
chmod 0700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 0644 ~/.ssh/authorized_keys

Step 3 - Rsync files over

Rsync is a great utility, as it allows you, among many other things, to copy files recursively with compression, and over an encrypted channel.

We will copy a file from our origin server (198.211.117.101) in /root/bigfile.txt over to our destination server (IP: 198.211.117.129) and save it in /root/bigfile.txt as well.

Login on 198.211.117.101 and rsync the file over to 198.211.117.129:

rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress /root/bigfile.txt 198.211.117.129:/root/

If you are using a different user, for example "username" then you would have to append it in front of destination server. Make sure to have your public key in that user's ~/.ssh/authorized_keys file:

rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress /root/bigfile.txt username@198.211.117.129:/

The SSH options are useful to keep Rsync quiet and not prompting everytime you connect to a new server.

Verify that you have received the file on destination server (198.211.117.129):

ls -la /root/bigfile.txt

And you are all done!

By Bulat Khamitov

28 Comments

  • I'm having issues getting this to work. I tried sending to /root/ and also to a different user. When I send to root, I get this error: rsync: change_dir#1 "/root/" failed: Permission denied (13) When I try to send it to user@IP it thinks I'm sending it locally and creates a directory on my machine. So I tried sending it to: ssh://user@IP and also tried ssh://IP:/root/ -- in both cases, it seems to get stuck. No errors, just stuck. Any suggestions on what to try next are appreciated in advance!
  • @scdavis41 what exact command are you running?
  • Here's the command I'm running in trying to send it to root: rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress ~/rocktree/dev/apps/countdown-timer/index.php 192.81.208.45:/root/ And here is the full error: rsync: change_dir#1 "/root/" failed: Permission denied (13) rsync error: errors selecting input/output files, dirs (code 3) at main.c(562) [Receiver=3.0.9] rsync: connection unexpectedly closed (8 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]
  • To be able to upload to /root, you have to connect as root: rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress ~/rocktree/dev/apps/countdown-timer/index.php root@192.81.208.45:/root/
  • Awesome! That did it. Thanks so much, Kamal.
  • @scdavis41: Glad I could help! :]
  • nice! thanks! '">xyz<1(
  • thanks !!
  • how to run rsync over ssh in a crontab... im having this problem ..do you have any suggestions ?? i gave full path to rsync ,,ssh etc etc ..the script works fine when executed but when place in crontab i doesnt work at all.
  • @urgen: Please paste the line you added to your crontab file.
  • Hope other locate's server becoming RAW format soon so that we can use the transfer function in DigitalOcean directly.
  • When I run ssh-keygen -f ~/.ssh/id_rsa -q -P "" it asks me if I want to override ~/.ssh/id_rsa? I am already using a SSH key on the server when created. I tried renaming id_rsa to id_rsa0 but that would not work. Is it possible to have multiple SSH keys or what am I doing wrong? Any help would be great! Thanks
  • @mcpackin.dev: If you already have a key set up, you don't need to regenerate it -- use your existing public key.
  • Hi there Thanks for the tutorial, it works like a charm.... At the risk of sounding like a total noob I wonder if you could explain the use of "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" Id love to know what this means/does?
  • @asparknz: These options stop SSH from checking the host keys file (~/.ssh/known_hosts) and from writing to it (by telling it to write to /dev/null instead). SSH stores each host's fingerprint when you connect to the said host for the first time and checks it on every subsequent connection. This makes sure that you're connecting to the same host and are not a victim of a MITM attack as each fingerprint is unique to one server only.
  • Hmmm, still asking for a password for the origin server. Here's what I'm running: rsync --ignore-existing -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress --remove-source-files user@originserver.gov:/home/user/folder/* /home/user/folder/
  • pls help:

    using rsync as mentioned results in ssh requests password on new/next session.

    Repro:

    • Fresh Ubuntu droplet with 512mb using existing ssh key
    • ssh into new droplet (works - as well as "exit" and reconnect with initial ssh key
    • using rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress ~/projects/vps-setup/dotfiles/ root@DROPLET-IP:/root/works fine, all files copied
    • exit current ssh session and reconnect using ssh root@DROPLET-IP keeps asking for the ssh password - even providing the associated password won´t allow me ssh'ing back into my droplet

    if I copy all files to /root/bin/ and use cp -r /root/dotfiles/.??* /root everything works fine, however, that can't be the solution...

    Any hints?

  • I did all that but it still keep asking me for password.
    I used this command:

    rsync -avz -e "ssh -p 1234 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress /home/dest/file.txt root@10.0.0.0:/home/dest/
    

    I've a custom SSH port defined by -p

  • @vago:
    Please check and let me know:

    -- In case you are also asked for a password running ssh root@yourip please try:
    ssh-keygen -R yourip (IPonly = no 'user@")
    (enter IP only) - that will remove the binding.
    -- Entering now again ssh root@yourip will ask you if you want to add your SSH key, choose yes, enter your password when you are asked for it.
    -- From there also your rsync command should run as expected.

    Also:
    In you example above you are having the IP '10.0.0.0', make sure you are using the correct ip.

  • i'm using rsync ,and going to delete on same directory ,it rises an error as denied

  • very useful Thanks

  • Can you update this tutorial to make it explicit which shell commands are issued on the virtual host and which are entered on the users local pc?

    • Its pretty much all in the text, that must be read to understand the commands provided...

      Please don't get me wrong, if this already is hard to understand, I really recommend getting help from a consultant/freelancer/sysadmin prior running anything important on a box as that box is very likely easy to compromise without performing much more complex operations than those described here.

      However, these are the words you are looking for:

      • origin = local system or also a local server that system (that holds the files you want to make available on another system (in this example the origin got the following IP: 198.211.117.101)
      • destination = remote server (aka droplet - in this example got the IP 198.211.117.129)
      • Thanks, have since built out a better understanding around all of this (at the time was setting up the Droplet as a sandbox to update my admin skills).

  • Had trouble with all the help here but I did find this site http://archive.oreilly.com/pub/h/38

    This worked for me:
    //copied bigfile.text in c:/root/ to server(once logged in of course)
    rsync -ave ssh /root/ /root/bigfile.txt/

  • I'm having the same issue. It keeps asking for the password.
    I'm on a windows 8.1 machine using cwRsync5.4.1x86_Free to generate my keys and rsync my files.

  • Super-useful and works like a charm, thank you very much.

  • I have access from my pc to both of my droplets with ssh and putty. At my pc I have the public and the private (.ppk) ssh keys. This works fine.

    Now I want both droplets to rsync in private network. I assueme I can still use the public and private keys I also use on pc in putty.
    But where do I put public and private key, so the first droplet knows that he can use this ssh keys to connect to droplet2 ?

    In this article I did not saw anything about the private key?!

Creative Commons License