Rsync between Mac and Linux

99.999999% of my readers can probably ignore this one, but if you are of the small minority who use Rsync and have Mac and Linux computers in your home you’ll want to read this.

I have Plex running on a Raspberry Pi for my music. I have a large mp3 folder. Too large to run Syncthing on it unfortunately, but an occasional rsync is perfectly fine.

I thought it worked fine until I quickly realised it was syncing the same files over and over again. It turns out the Mac and Linux machines I’m using have different ideas about character sets their filenames are stored in. A file with an accented character on the Mac is completely different to one that looks the same on the Linux box.

The solution took a while for me to find but it’s very simple. Rsync has an option named --iconv to convert between character sets!

The solution was embarrassingly simple: Much due to a comment I read when researching the problem, I thought you were supposed to specify the character set in the order of transformation; but it seems as that is not the correct syntax. Rather, one should always use --iconv=utf-8-mac,utf-8 when initialising the rsync from the mac, and always use --iconv=utf-8,utf-8-mac when initialising the rsync from the linux machine, no matter if I want to sync files from the mac or linux machine.

Then it works like magic!

EDIT: Indeed, sometimes, checking the manual page closely is a good thing to do. Here it is, black on white:

              Rsync  can  convert  filenames between character sets using this
              option.  Using a CONVERT_SPEC of "." tells rsync to look up  the
              default  character-set via the locale setting.  Alternately, you
              can fully specify what conversion to do by giving a local and  a
              remote   charset   separated   by   a   comma   in   the   order
              --iconv=LOCAL,REMOTE, e.g.  --iconv=utf8,iso88591.   This  order
              ensures  that the option will stay the same whether you're push-
              ing  or  pulling  files.

4 thoughts on “Rsync between Mac and Linux

  1. This does only work from mac to *nux as the libiconv developer refuces to accept the change which would give all platforms utf-8-mac …

    The way he either responds or does not responds to these requests shows that he probably has a personal vendetta towards Apple.

    1. Casper – What do you use for syncing with Windows machines? Many years ago I did rsync between Windows and Linux machines but haven’t done so since. The closest I got to was using Syncthing to sync between a Windows machine (source) and Mac (destination). It mostly worked except it stalled with about 5GB of data refusing to sync.

      In the end I used Filezilla to sftp the directories I wanted but it refused to create empty directories. To be extra safe I also copied the files to an NTFS formatted external SSD. I’ll go through them eventually and see what’s missing.

  2. Thank you, your advice is incredibly helpful. I have lived with the problem of rsynch failing to work on music files containing accented characters for some years. I must indeed be one of the .000001% of the readers you mention will be interested in this post!

    1. Just to add, for anyone else’s benefit, that you need to have a recent version of rsynch installed for this to work. The one that comes with any Mac has a pre-installed old version that doesn’t support the –iconv option. Updating Rsynch to a current version solves that problem..

Leave a Reply

%d bloggers like this:

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.