Dir | Technology Tales

TOPIC: DIR

Something to try when you get a message like this caused by a filename with a leading hyphen: "mv: illegal option -- u"

3^rd December 2024

Recently, I downloaded some WEBP files from Ideogram and attempted to move them to another folder. That is when I got the message that you see in the title of this entry. Because I had not looked at the filenames, I baffled when I got this from a simple command that I had been using with some success until then. Because I was using an iMac, I tried the suggestion of installing coreutils to get GNU mv and cp to see if that would help:

brew install coreutils

The above command gave me gmv and gcp for the GNU versions of mv and cp that comes with macOS. Trying gmv only got me the following message:

gmv: cannot combine --backup with --exchange, -n, or --update=none-fail

The ls command could list all files, but not the WEBP ones. Thus, I executed the following to show what I wanted:

ls | grep -i webp

That got around the problem by doing a subset of the directory listing. It was then that I spotted the leading hyphen. To avoid the problem tripping me up again, I renamed the offending file using this command:

mv -- -iunS9U4RFevWpaju6ArIQ.webp iunS9U4RFevWpaju6ArIQ.webp

Here, the -- switch tells the mv command not to look for any more options and only to expect filenames. When I tried enclosing the filename in quotes, I still got problems, even that might have because I was using double quotes instead of single quotes. Another option is to escape the leading hyphen like this:

mv ./-iunS9U4RFevWpaju6ArIQ.webp iunS9U4RFevWpaju6ArIQ.webp

Once the offending file was renamed, I could move the files to their final location. That could have used the -- option too, saving me an extra command, only for my wanting this not to trip me up again. Naturally, working in Finder might have avoided all this as much as not having a file with a leading hyphen in its name, but there would have been nothing to learn then.

Copying a directory tree on a Windows system using XCOPY and ROBOCOPY

17^th September 2016

My usual method for copying a directory tree without any of the files in there involves the use of the Windows command line tool XCOPY and the command takes the following form:

xcopy /t /e <source> <destination>

The /t switch tells XCOPY to copy only the directory structure, while the /e one tells it to include empty directories too. Substituting /s for /e would ensure that only non-empty directories are copied. <source> and <destination> are the directory paths that you want to use and need to be enclosed in quotes if you have a space in a directory name.

There is one drawback to this approach that I have discovered. When you have long directory paths, messages about there being insufficient memory are issued and the command fails. The limitation has nothing to do with the machine that you are using, but is a limitation of XCOPY itself.

After discovering that, I got to check if ROBOCOPY can do the same thing without the same file path length limitation because I did not have the liberty of shortening folder names to get the whole path within the length expected by XCOPY. The following is the form of the command that I found did what I needed:

robocopy <source1> <destination1> /e /xf *.* /r:0 /w:0 /fft

Here, <source1> and <destination1> are the directory paths that you want to use and need to be enclosed in quotes if you have a space in a directory name. The /e switch copies all subdirectories and not just non-empty ones. Then, the xf *.* portion excludes all files from the copying process. The remaining options are added to help with getting around access issues and to try to copy only those directories that do not exist in the destination location. The /ftt switch was added to address the latter by causing ROBOCOPY to assume FAT file times. To get around the folder permission delays, the /r:0 switch was added to stop any operation being retried, with /w:0 setting wait times to 0 seconds. All this was enough to achieve what I wanted, and I am keeping it on file for my future reference, as well as sharing it with you.

Restoring GNU Parallel Functionality in Ubuntu GNOME 13.04

31^st July 2013

There is a handy command line utility called GNU Parallel that allows you to run Linux commands on more than one CPU core at a time to perform parallel processing of the task at hand. Here is a form of the command that is similar to one that I often use:

ls *.* | parallel gm convert -sharpen 1x3 {} sharpened_images/{}

What it does is pipe a list of files in a folder to GraphicsMagick for sharpening and outputting to a sharpened_images directory. The {} in the command is where the filenames go in the sharpening command.

This worked fine in Ubuntu GNOME 12.10 but stopped doing so after I upgraded to the next version. A look on the web set me to running the following command:

parallel --version

That produced output that included the following line:

WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.

Rerunning the original command with the --gnu option worked, but there was a more permanent solution than using something like this:

ls *.* | parallel --gnu gm convert -sharpen 1x3 {} sharpened_images/{}

That was editing /etc/parallel/config with root privileges to delete the --tollef option from there. With that completed, all was as it should again, and it makes me wonder why the change was made in the first place. Perhaps because of it, there even is a discussion about the possibility of removing the --tollef option altogether, since it raises more questions than it answers.

Using SAS FILENAME statement to extract directory file listings into SAS

30^th May 2007

The filename statement's pipe option allows you to direct the output of operating system commands into SAS for further processing. Usefully, the Windows dir command (with its /s switch) and the UNIX and Linux equivalent ls allow you to get a file listing into SAS. For example, here's how you extract the list of files in your UNIX or Linux home directory into SAS:

filename DIRLIST pipe 'ls ~';
data dirlist;
    length filename $200;
    infile dirlist length=reclen;
    input buffer $varying200. reclen;
run;

Using the ftp option on the filename statement allows you to get a list of the files in a directory on a remote server, even one with a different operating system to that used on the client (PC or server), very useful for cases where cross-platform systems are involved. Here's some example code:

filename dirlist ftp ' ' ls user='user' host='host' prompt;
data _null_;
    length filename $200;
    infile dirlist length=reclen;
    input buffer $varying200. reclen;
run;

The PROMPT option will cause SAS to ask you for a password, and the null string is where you would otherwise specify the name of a file.