rsync local files to webserver

(this is a revision of a post I made on 12/5/12 on namecheap’s support forum)

I wanted to use rsync to upload/synchronize web site files from my local computer to my public_html directory on my hosting account at namecheap.

here are some things I found helpful:

SSH

I’m using the SSH protocol in rsync, rather than the rsync protocol. Make sure you can ssh into your server and I recommend setting up password-free ssh. (On namecheap, you have to request SSH access with a help ticket or chat)

rsync installed on remote server

rsync has to be installed on your remote server, in the path and executable when you ssh in.

A note on rsync: When you call rsync locally over ssh, it starts an rsync process on the remote server in order to coordinate the transfer. This is why rsync has to be executable on the server, even if you are only uploading to the server. that’s what I understand, at least.

You should verify that rsync is available by connecting via ssh and running:

$ which rsync

If this comes back with a path to rsync, you should be golden. if it comes back with “command not found”, you need to install rsync and/or add it to your path on the remote. (for namecheap, you need to submit a ticket to request rsync).

rsync test

You can run a quick test, which will not modify anything on your host, with this:

$ echo "hi" > test_file
$ rsync --dry-run -v -e "ssh -p 21098" test_file [user]@[your-url].com:~/public_html

This should come back with a transfer report. However, the –dry-run flag just checks what *would* be transferred without moving anything. the -e flag specifies the ssh setup. You’ll need ssh protocol if you didn’t set up an rsync deamon on your remote server.

ssh port

My Namecheap hosting uses port 21098 for ssh, which is why I have the ssh parameter:

-e "ssh -p 21098"

Your ssh may use a different port. If you can ssh successfully from the command line without specifying the port, than you are using the default port and do not need this -p parameter above. If you do require a custom ssh port but either forget this parameter or enter the port wrong, the rsync command will hang without feedback.

Final setup

Here’s the final setup I use to mirror a local directory to a public_html folder. This is broken out and commented, you’ll want to clean it up to one line without my comments:

$ rsync
 -avz                  ## use archive mode, verbose feedback and compression
 -e 'ssh -p 21098'     ## use ssh protocol with namecheap's port
 --dry-run             ## include this for testing, remove this to actually transfer
 --delete              ## delete files on the destination (namecheap server)
                       ## that don't exist locally. know what you are doing
                       ## if you include this flag
 --stats               ## provide extra info, delete if you don't want this
 --progress            ## provide extra info, delete if you don't want this
 --exclude ".DS_Store" ## I'm on a mac, don't include this standard hidden file
 --exclude ".htaccess" ## don't overwrite my .htaccess files on the server
 --exclude ".ftpquota" ## don't overwrite ftp quota files on the server
 source_dir/           ## this is the source folder on my computer.
                       ## Use the trailing slash to copy it's contents, not the directory
 [user]@[your-url].com:~/public_html/dest_dir/ ## destination directory

Troubleshooting

If you are having trouble with the initial rsync test, you can put ssh in verbose mode and possibly see where the problem is. add -v inside the quoted parameter to -e

$ echo "hi" > test_file
$ rsync --dry-run -v -e "ssh -v -p 21098" test_file [user]@[your-url].com:~/public_html

rsync *every* file the first time?

If you’ve been using a different solution to transfer existing local files to your server (e.g. ftp), you may find that the first time you call rsync, it will re-upload *every* file.

WTF, rsync is only suppose to do incremental differences, right?

I believe what is happening is this: rsync notices that the timestamps for your local and remote copies are different. Your local was made at 12:30 PM and then you uploaded at 12:31 PM, so your remote has a later timestamp and is therefore considered a different file. The first time you run rsync, it is going to push your local 12:30 PM file to the server but preserve the 12:30 PM timestamp because you are using the -a archive flag, regardless of what time the transfer itself occurs. Now as if you rerun rsync, it won’t do this again because all the files match. For here on, incremental changes only.

 

One thought on “rsync local files to webserver

  1. Pingback: Namecheap shared hosting starting guide | karine.do/writes

Leave a Reply

Your email address will not be published.