Categories
Linux

Rsync between Mac and Linux

99.999999% of my readers can probably ignore this one, but if you are of the small minority who use Rsync and have Mac and Linux computers in your home you’ll want to read this.

I have Plex running on a Raspberry Pi for my music. I have a large mp3 folder. Too large to run Syncthing on it unfortunately, but an occasional rsync is perfectly fine.

I thought it worked fine until I quickly realised it was syncing the same files over and over again. It turns out the Mac and Linux machines I’m using have different ideas about character sets their filenames are stored in. A file with an accented character on the Mac is completely different to one that looks the same on the Linux box.

The solution took a while for me to find but it’s very simple. Rsync has an option named --iconv to convert between character sets!

The solution was embarrassingly simple: Much due to a comment I read when researching the problem, I thought you were supposed to specify the character set in the order of transformation; but it seems as that is not the correct syntax. Rather, one should always use --iconv=utf-8-mac,utf-8 when initialising the rsync from the mac, and always use --iconv=utf-8,utf-8-mac when initialising the rsync from the linux machine, no matter if I want to sync files from the mac or linux machine.

Then it works like magic!

EDIT: Indeed, sometimes, checking the manual page closely is a good thing to do. Here it is, black on white:

--iconv=CONVERT_SPEC
              Rsync  can  convert  filenames between character sets using this
              option.  Using a CONVERT_SPEC of "." tells rsync to look up  the
              default  character-set via the locale setting.  Alternately, you
              can fully specify what conversion to do by giving a local and  a
              remote   charset   separated   by   a   comma   in   the   order
              --iconv=LOCAL,REMOTE, e.g.  --iconv=utf8,iso88591.   This  order
              ensures  that the option will stay the same whether you're push-
              ing  or  pulling  files.

By Donncha

Donncha Ó Caoimh is a software developer at Automattic and WordPress plugin developer. He posts photos at In Photos and can also be found on Twitter.

2 replies on “Rsync between Mac and Linux”

This does only work from mac to *nux as the libiconv developer refuces to accept the change which would give all platforms utf-8-mac …

The way he either responds or does not responds to these requests shows that he probably has a personal vendetta towards Apple.

Casper – What do you use for syncing with Windows machines? Many years ago I did rsync between Windows and Linux machines but haven’t done so since. The closest I got to was using Syncthing to sync between a Windows machine (source) and Mac (destination). It mostly worked except it stalled with about 5GB of data refusing to sync.

In the end I used Filezilla to sftp the directories I wanted but it refused to create empty directories. To be extra safe I also copied the files to an NTFS formatted external SSD. I’ll go through them eventually and see what’s missing.

Leave a Reply