Adventures & experiences in contemporary technology
With a growing collection of photographic images, I often find myself making backups of files using copy commands and the data volumes are such that I don’t want to keep copying the same files over and over again so incremental file transfers are what I need. So commands like the following often get issued from a Linux command line:
cp -pruv [source] [destination]
Because this is in Linux, it the bash shell that I use so the switches may not apply to others like ssh, fish or ksh. For my case, p preserves file properties such as its time and date and the cp command does not do this always so it needs adding. The r switch is useful because the copy then in recursive so only a directory needs to specified as the source and the destination needs to be one level up from a folder with the same name there so as to avoid file duplication. It is the u switch that makes the file copy incremental and the v one issues messages to the shell that show how the copying is going. Seeing a file name issued by the latter does tell you how much more needs to be copied and that the files are going where they should.
What inspired this post though is my need to do the same in a Windows session and issuing xcopy commands will achieve the same end. Here are two that will do the needful:
xcopy [source] [destination] /d /s
xcopy [source] [destination] /d /e
In both cases, it is the d switch that ensures that the copy is incremental and you can add a date too, with a colon between it and the /d, if you see fit. The s switch copies only directories that contain files while the e one copies even empty directories. Using the d switch without either of those did not trigger any copying action when I tried so I reckon that you cannot do without either of them. By default, both of these commands issue output to the command line so you can keep an eye on what is happening and this especially is useful when ensuring that files are going to the right destination because the behaviour differs from that of the bash shell in Linux.
The changes made to Nautilus, otherwise known as Files, in GNOME Shell 3.6 were contentious and the response of the Linux Mint was to create their own variant called Nemo from the previous version of the application. On the Cinnamon or MATE desktop environments, the then latest version of GNOME’s file manager would have looked like a fish out of water without its application menu in the top panel on the GNOME Shell desktop. It is possible to make a few modifications that help Nautilus to look more at home on those Linux Mint desktops and I have collected them here because they are useful for GNOME Shell users too. Here they are in turn.
Adding Application Menu entries to Location Options Menu
The Location Options menu is what you get on clicking the button with the cog icon on the right-hand side of the application’s location bar. Using Gsettings, it is possible to make that menu include the sort of entries that are in the application menu in the GNOME Shell panel at the top of the screen. These include an entry for closing the whole application as well as setting its preferences (or options). Running the following command does just that (if it does not work as it should, try changing the single and double quotes to those understood by a command shell):
gsettings set org.gnome.settings-daemon.plugins.xsettings overrides '@a{sv} {"Gtk/ShellShowsAppMenu": <int32 0>}'
Adding in the Remove App Menu GNOME Shell extension will clean up the GNOME Shell a little by removing the application menu altogether. If, for some reason, you wish to restore the default behaviour, then the following command does the required reset:
gsettings set org.gnome.settings-daemon.plugins.xsettings overrides '@a{sv} {}'
Stopping Hiding of the Application Title Bar When Maximised
By default, GNOME Shell can hide the application title bars of GNOME applications such as Nautilus on window maximisation and this is Nautilus now works by default. Changing the behaviour so that the title bar is kept on maximised windows can be as simple as adding in the ignore_request_hide_titlebar extension. The trouble with GNOME Shell extensions is that they can stop working when a new version of GNOME Shell is used, so there’s another option: editing metacity-theme-3.xml but /usr/share/themes/Adwaita/metacity-1. The file can be opened using superuser privileges using the following command:
gksudo gedit /usr/share/themes/Adwaita/metacity-1/metacity-theme-3.xml
With the file open, it is a matter of replacing instances of ' has_title="false" ' with ' has_title="true" ', saving it and reloading GNOME Shell. This may persevere across different versions of GNOME Shell should the extension not do so.
Disabling Recursive Search
This discovery is what led me to bundle these customisations in an entry on here in the first place. In Nemo and older versions of Nautilus, just typing with the application open would lead you down a list towards the file that you wanted. This behaviour was replaced by an automatic recursive search from GNOME Shell 3.6 where the search functionality was extended beyond the folder that was open in the file manager to its subdirectories. To change that to subsetting within the open folder or directory, you need to install a patch version of Nautilus using the following commands:
sudo add-apt-repository ppa:dr3mro/personal
sudo apt-get update && sudo apt-get upgrade
The first of these adds a new repository with the patched version of Nautilus while the second combination installs the patched version. With that done, it is time to issue the following command:
gsettings set org.gnome.nautilus.preferences enable-recursive-search false
That sets the value of the new enable-recursive-search option to false for searching within an open directory. It also can be found using Dconf-Editor in the following hierarchy: org -> gnome -> nautilus -> preferences. The obsession of the GNOME project team with minimalism is robbing users of some options and this would be a good one to have by default too. Maybe the others should be treated in the same way even if you need to use Gsettings or Dconf-Editor to change them to avoid clutter. Having GNOME Tweak Tool able to set them all would be even better.
There is a bill making its way through the U.K. parliament at the moment that could reduce the power of copyright when it comes to images placed on the web. The current situation is that anyone who creates an image automatically holds the copyright for it. However, the new legislation will remove that if it becomes law as it stands. As it happens, the Royal Photographic Society is doing what it can to avoid any changes to what we have now. There may be the barrier of due diligence but how many of us take steps to mark our own intellectual property? For one, I have been less that attentive to this and now wonder if there is anything more that I should be doing. Others may copyleft their images but I don’t want to find myself unable to share my own photos because another party is claiming rights over them. There’s watermarking them but I also want to add something to the image metadata too.
That got me wondering about adding metadata to any images that I post online that assert my status as the copyright holder. It may not be perfect but any action is better than doing nothing at all. Given that I don’t post photos where EXIF metadata is stripped as part of the uploading process, it should be there to see for anyone who bothers to check and there may not be many who do.
Because I also wanted to batch process images, I looked for a command line tool to do the needful and found ExifTool. Being a Perl library, it is cross-platform so you can use it on Linux, Windows and even OS X. To install it on a Debian or Ubuntu based Linux distro, just use the following command:
sudo apt-get install libimage-exiftool-perl
The form of the command that I found useful for adding the actual copyright information is below:
exiftool -p “-copyright=(c) John …” -ext jpg -overwrite_original
The -p switch preserves the timestamp of the image file while the -overwrite_original one ensures that you don’t end up with unwanted backup files. The copyright message goes within the quotes along with the -copyright option. With a little shell scripting, you can traverse a directory structure and change the metadata for any image files contained in different sub-folders. If you wish to do more than this, there’s always the user documentation to be consulted.
Linux and UNIX have the touch command for changing the creation dates and times for files. However, it also will create empty text files for you as well. In fact, there are times when I feel the need to do this sort of thing on Windows too and the following command accomplishes the deed when run in a Command Prompt window:
type nul > command.bat
Essentially, null output is sent to a file that is created anew, command.bat in this case. Then, you can edit it in Notepad (or whatever is your choice of text editor) and add in what you need. This will not work in Powershell so you need another command for that:
New-Item command.bat -type file
This uses the New-Item command, which also can be used to create folders as well if you so desire. Then, the command becomes the following:
New-Item c:\commands -type directory
Note that file on the previous example has become directory and there is the -force option should you need to overwrite what already exists for some reason…
That other use of the UNIX/Linux touch command can be performed from the Command Prompt too and here is an example command:
copy /b file.txt +,,
The /b switch switches on binary behaviour for the copy command though that appears to be the default action anyway. The + operator triggers concatenation and ,, gets around not having a defined destination because you cannot copy a file over itself. If that were possible, then there would no need for special syntax for changing the date and time for a file.
For doing the same thing with Powershell, try the following:
(GetChildItem test.txt).LastWriteTime=Get-Date
The GetChildItem command has aliases of gci, dir and ls and the last two of these give away its essential purpose. Here, it is used to pick out the test.txt file so that its timestamp can be replaced with the current date and time returned by the Get-Date command. The syntax looks a little more complex even if it achieves the same end. Somehow, that touch command is easier to explain. Are Linux and UNIX that complicated after all?
The Linux and UNIX command mv has a number of limitations, such as not overwriting destination files and not renaming multiple files using wildcards. The only solution to the first that I can find is one that involves combining the cp and rm commands. For the second, there’s another command: rename. There is a command like it in Windows but this one is a little different in its syntax. Before saying more about that, here’s an example like what I used recently:
rename s/fedora/fedora2/ fedora.*
The first argument in the above command is a regular expression much like what Perl is famous for implementing; in fact, it is Perl-compatible ones (PCRE) that are used. The s before the first slash stands for substitute with fedora being the string that needs to be replaced and fedora being what replaces it. The third command is the file name glob that you want to use, fedora.* in this case. Therefore, all files in a directory named fedora will be renamed fedora2 regardless of the file type. The same sort of operation can be performed for all files with the same extension and it needing changing, htm to html, for instance. Of course, there are other uses but these are handy ones to know.
So as not to deface anything that is available online on the web, I have a tendency to set up an offline Apache server on a home PC to do any tinkering away from the eyes of the unsuspecting public. Though Ubuntu is my mainstay for home computing, I do have a PC with Fedora installed and I have been trying to get an Apache instance starting automatically on there without success for a few months. While I can start it by running the following command as root, I’d rather not have more manual steps than is necessary.
httpd -k start
The command used by the system when it starts is different and, even when manually run as root, it failed with messages saying that it couldn’t find the directory while the web server files are stored. Here it is:
service httpd start
The default document root location on any Linux distribution that I have seen is /var/www and all is very well with this but it isn’t a safe place to leave things if ever a re-installation is needed. Having needed to wipe /var after having it on a separate disk or partition for the sake of one installation, it doesn’t look so persistent to me. In contrast, you can safeguard /home by having it on another disk or in a dedicated partition and it can be retained even when you change the distro that you’re using. Thus, I have got into the habit of having the root of the web server document root folder in my home area and that is where I have been seeing the problem.
Because of the access message, I tried using chmod and chgrp but to no avail. The remedy has to do with reassigning the security contexts used by SELinux. In Fedora, Apache will not work with the context user_home_t that is usually associated with home directories but needs httpd_sys_content_t instead. To find out what contexts are associated with particular folders, issue the following command:
ls -Z
The final solution was to create a user account whose home directory hosts the root of the web server file system, called www in my case. Then, I executed the following command as root to get things going:
chcon -R -h -t httpd_sys_content_t /home/www
It seems that even the root of the home directory has to have an appropriate security context (/home has home_root_t so that might do the needful too). Without that, nothing will work even if all is well at the next level down. The switches for chcon command translate as follows:
-R : recursive; applies changes to all files and folders within a directory.
-h : changes apply only to symbolic links and not to where they refer in the file system.
-t : alters context type.
It took a while for all of this stuff about SELinux security contexts to percolate through to the point where I was able to solve the problem. A spot of further inspiration was needed too and even guided my search for the information that I needed. It’s well worth trying Linux Home Networking if you need more information. There are references to an earlier release of Fedora but the content still applies to later versions of Fedora right up to the current release if my experience is typical.
These times, my virtualisation needs are being well served by VirtualBox 2.2. It may be the closed source variant but I have no complaints about it. Along with a number Windows VM’s, I also have one running Ubuntu 9.04 and, for the first time, I seem to have VirtualBox’s Guest Additions playing with a Linux guest as they should. Even the Shared Folders functionality is working.
However, I did get one problem when I tried out the last feature for the first time. The procedure is to issue a command like the following in a terminal session after creating the requisite directory in the file system and adding a host directory as a shared folder:
sudo mount -t vboxsf Music /mnt/host_music/
Above, Music is the name of the folder in the VirtualBox manager and /mnt/host_music in the directory in the guest file system. However, this returned the message at the head of this post at that first attempt:
/sbin/mount.vboxsf: mounting failed with the error: Protocol error
The solution thankfully turns out to be an easy one: reinstalling the Guest Additions and that certainly did the trick for me. The cause would appear to have been an update to Ubuntu and 9.04 is understandably in a state of flux at the moment (I suspect kernel upgrades because of my previous experiences). Regardless of this, it is good to know that it’s a problem with a simple fix and I am seeing the niceties of a larger virtual screen system together automatic grabbing and releasing of the mouse cursor too. There may be a chance to explore the availability of these sorts of features to other Linux guests but I have other things that I should be doing and there’s sunshine outside to be enjoyed.
Here are a few UNIX commands that I have recently encountered that help with process management and are particularly useful when jobs are running in the background. Here they are:
nohup
It’s short for no hang up and stops termination of a job when a user logs off. Another result is that all console messages being directed to a file called nohup.out in the directory current to the job being run, or in the user’s home directory where write access to the current working directory is unavailable.
ps
This returns a list of processes, their ID’s and their statuses. By default, this is for your own processes but you can look beyond this with the myriad of options that can be passed. For instance, the -U switch allows you to look at a job for other users while the -f one shows more information than the standard call and this even includes the commands submitted to start the ongoing processes.
kill
The name says it all and it’s far quicker than the rigmarole that you have to endure with the Windows task manager; I wonder if there is a command line approach to process termination on Windows.
The filename statement’s pipe option allows you to direct the output of operating system commands into SAS for further processing. Usefully, the Windows dir command (with its /s switch) and the UNIX and Linux equivalent ls allow you get a file listing into SAS. For example, here’s how you extract the list of files in your UNIX or Linux home directory into SAS:
filename DIRLIST pipe 'ls ~';
data dirlist;
length filename $200;
infile dirlist length=reclen;
input buffer $varying200. reclen;
run;
Using the ftp option on the filename statement allows you to get a list of the files in a directory on a remote server, even one with a different operating system to that used on the client (PC or server), very useful for cases where cross-platform systems are involved. Here’s some example code:
filename dirlist ftp ' ' ls user='user' host='host' prompt;
data _null_;
length filename $200;
infile dirlist length=reclen;
input buffer $varying200. reclen;
run;
The PROMPT option will cause SAS to ask you for a password and the null string is where you would otherwise specify the name of a file.