Adventures & experiences in contemporary technology
The rsync command is one that I use heavily for doing backups and web publishing. The latter means that it is part of how I update websites built using Hugo because new and/or updated files need uploading. The command also sees usage when uploading files onto other websites as well. During one of these operations, and I am unsure now as to which type is relevant, I encountered errors about being unable to set permissions.
The cause was the encompassing -a option. This is a shorthand for -rltpgoD, and the individual options perform the following:
-r: recursive transfer, copying all contents within a directory hierarchy
-l: symbolic links copied as symbolic links
-t: preserve times
-p: preserve permissions
-g: preserve groups
-o: preserve owners
-D: preserve device and special files
The solution is to some of the options if they are inappropriate. The minimum is to omit the option for permissions preservation, but others may not apply between different servers either, especially when operating systems differ. Removing the options for preserving permissions, groups and owners results in something like this:
rsync -rltD [rest of command]
While it can be good to have a more powerful command with the setting of a single option, it can mean trying to do too much. Another way to avoid permissions and similar errors is to have consistency between source and destination files systems, but that is not always possible.
In the world of UNIX and Linux, symbolic links are shortcuts but they do not work like normal Windows shortcuts because you do not jump from one location to another with the file manager’s address bar changing what it shows. Instead, it is as if you see the contents of the directory at another quicker to access location in the file system and the same sort of thinking applies to files too. In some ways, it is like giving files and directories alternative aliases. There are soft links that point to the name of a given directory or file and hard links that point to actual files or directories.
For a long time, I was under the mistaken impression that such things did not exist in Windows until I came across the mklink command, which came with the launch of Windows Vista at the start of 2007. While the then new feature may not be an obvious to most of us, it does does that Windows did incorporate a feature from UNIX and Linux well before the advent of virtual desktops on Windows 10.
By default, the aforementioned command sets up symbolic links to files and the /D switch allows the same to be done for directories too. The /H switch makes a hard link instead of a soft link so we get much of the functionality of the ln command in UNIX and Linux. Here is an example that creates a soft symbolic link for a directory:
mklink /D shortcut target_directory
Above, shortcut is the name of the symbolic link file and target_directory is the destination to which it links. In my experience, it works best for destinations beyond your home folder and, from what I have read, hard links may not be possible across different disks either.
The official line from Western Digital is that they do not support the use of their My Book Live NAS drives with Linux or UNIX. However, what that means is that they only develop tools for accessing their products for Windows and maybe OS X. It still doesn’t mean that you cannot access the drive’s configuration settings by pointing your web browser at http://mybooklive.local/. In fact, not having those extra tools is no drawback at all since the drive can be accessed through your file manager of choice under the Network section and the default name is MyBookLive too so you easily can find the thing once it is connected to a router or switch anyway.
Once you are in the servers web configuration area, you can do things like changing its name, updating its firmware, finding out what network has been assigned to it, creating and deleting file shares, password protecting file shares and other things. These are the kinds of things that come in handy if you are going to have a more permanent connection to the NAS from a PC that runs Linux. The steps that I describe have worked on Ubuntu 12.04 and 13.10 with the GNOME desktop environment.
What I was surprised to discover that you cannot just set up a symbolic link that points to a file share. Instead, it needs to be mounted and this can be done from the command line using mount or at start-up with /etc/fstab. For this to happen, you need the Common Internet File System utilities and these are added as follows if you need them (check on the Software Software or in Synaptic):
sudo apt-get install cifs-utils
Once these are added, you can add a line like the following to /etc/fstab:
//[NAS IP address]/[file share name] /[file system mount point] cifs
credentials=[full file location]/.creds,
iocharset=utf8,
sec=ntlm,
gid=1000,
uid=1000,
file_mode=0775,
dir_mode=0775
0 0
Though I have broken it over several lines above, this is one unwrapped line in /etc/fstab with all the fields in square brackets populated for your system and with no brackets around these. Though there are other ways to specify the server, using its IP address is what has given me the most success and this is found under Settings > Network on the web console. Next up is the actual file share name on the NAS and I have used a custom them instead of the default of Public. The NAS file share needs to be mounted to an actual directory in your file system like /media/nas or whatever you like and you will need to create this beforehand. After that, you have to specify the file system and it is cifs instead of more conventional alternatives like ext4 or swap. After this and before the final two space delimited zeroes in the line comes the chunk that deals with the security of the mount point.
What I have done in my case is to have a password protected file share and the user ID and password have been placed in a file in my home area with only the owner having read and write permissions for it (600 in chmod-speak). Preceding the filename with a “.” also affords extra invisibility. That file then is populated with the user ID and password like the following. Of course, the bracketed values have to be replaced with what you have in your case.
username=[NAS file share user ID]
password=[NAS file share password]
With the credentials file created, its options have to be set. First, there is the character set of the file (usually UTF-8 and I got error code 79 when I mistyped this) and the security that is to be applied to the credentials (ntlm in this case). To save having no write access to the mounted file share, the uid and gid for your user needs specification with 1000 being the values for the first non-root user created on a Linux system. After that, it does no harm to set the file and directory permissions because they only can be set at mount time; using chmod, chown and chgrp later on has no effect whatsoever. Here, I have set permissions to read, write and execute for the owner and the user group while only allowing read and execute access for everyone else (that’s 775 in the world of chmod).
All of what I have described here worked for me and had to gleaned from disparate sources like Mount Windows Shares Permanently from the Ubuntu Wiki, another blog entry regarding the permissions settings for a CIFS mount point and an Ubuntu forum posting on mounting CIFS with UTF-8 support. Because of the scattering of information, I just felt that it needed to all together in one place for others to use and I hope that fulfils someone else’s needs in a similar way to mine.
Linux Mint 14 may be out now but I’ll be sticking with its predecessor for now. Being a user of GNOME Shell instead of Cinnamon or Mate, I’ll wait for extensions to get updated for 3.6 before making a move away from 3.4 where the ones that I use happily work. Given that Linux Mint 13 is set to get support until 2017, it’s not as if there is any rush either. Adding the back-ported packages repository to my list of software sources means that I will not miss out on the latest versions of MDM, Cinnamon and Mate anyway. With Ubuntu set to stick to GNOME 3.6 until after 13.04 is released, adding the GNOME 3 Team PPA will be needed if 3.8 arrives with interesting goodies; there are interesting noises that suggest the approach taken in Linux Mint 12 may be used to give more of a GNOME 2 desktop experience. Options abound and there are developments in the pipeline that I hope to explore too.
However, there is one issue that I have had to fix which stymies upgrades within the 3.2 kernel branch. A configuration file (/etc/grub.d/10_linux) points to /usr/share/grub/grub-mkconfig_lib instead of /usr/lib/grub/grub-mkconfig_lib so I have been amending it every time I needed to do a kernel update. However, it just reverts to the previous state so I thought of another solution: creating a symbolic link in the incorrect location that points to the correct one so that updates complete without manual intervention every time. The command that does the needful is below:
sudo ln -s /usr/lib/grub/grub-mkconfig_lib /usr/share/grub/grub-mkconfig_lib
Of course, figuring out what causes the reversion would be good too but the symbolic link fix works so well that there’s little point in exploring it further. Of course, if anyone can add how you’d do that, I’d welcome this advice too. New knowledge always is good.
UNIX (and Linux) does a wonderful trick with its file and folder shortcuts; it effectively treats them as file and folder transporters that transfer associate a file or folder that exists in one folder hierarchy with another and it is treated as if it exists in that hierarchy too. For example, the images folder under /www/htdocs/blog can have a link under /www/htdocs/ that makes it appear that its contents exist in both places without any file duplication. For instance, the pwd command cannot tell a folder from a folder shortcut. To achieve this, I use what are called symbolic links and the following command achieves the outcome in the example:
ln -s /www/htdocs/blog/images /www/htdocs/images
The first file path is the destination for the link while the second one is that for the link itself. I had a problem with Google Reader not showing up images in its feed displays so symbolic links rode to the rescue as they did for resolve a similar conundrum that I was encountering when editing posts in my hillwalking blog.