TOPIC: GNU PARALLEL
Batch conversion of DNG files to other file types with the Linux command line
8th June 2016At the time of writing, Google Drive is unable to accept DNG files, the Adobe file type for RAW images from digital cameras. While the uploads themselves work fine, the additional processing at the end that, I believe, is needed for Google Photos appears to be failing. Because of this, I thought of other possibilities like uploading them to Dropbox or enclosing them in ZIP archives instead; of these, it is the first that I have been doing and with nothing but success so far. Another idea is to convert the files into an image format that Google Drive can handle, and TIFF came to mind because it keeps all the detail from the original image. In contrast, JPEG files lose some information because of the nature of the compression.
Handily, a one line command does the conversion for all files in a directory once you have all the required software installed:
find -type f | grep -i "DNG" | parallel mogrify -format tiff {}
The find and grep commands are standard, with the first getting you a list of all the files in the current directory and sending (piping) these to the grep command, so the list only retains the names of all DNG files. The last part uses two commands for which I found installation was needed on my Linux Mint machine. The parallel package is the first of these and distributes the heavy workload across all the cores in your processor, and this command will add it to your system:
sudo apt-get install parallel
The mogrify command is part of the ImageMagick suite along with others like convert and this is how you add that to your system:
sudo apt-get install imagemagick
In the command at the top, the parallel command works through all the files in the list provided to it and feeds them to mogrify for conversion. Without the use of parallel, the basic command is like this:
mogrify -format tiff *.DNG
In both cases, the -format switch specifies the output file type, with the tiff portion triggering the creation of TIFF files. The *.DNG portion itself captures all DNG files in a directory, but {} does this in the main command at the top of this post. If you wanted JPEG ones, you would replace tiff with jpg. Should you ever need them, a full list of what file types are supported is produced using the identify command (also part of ImageMagick) as follows:
identify -list format
Restoring GNU Parallel Functionality in Ubuntu GNOME 13.04
31st July 2013There is a handy command line utility called GNU Parallel that allows you to run Linux commands on more than one CPU core at a time to perform parallel processing of the task at hand. Here is a form of the command that is similar to one that I often use:
ls *.* | parallel gm convert -sharpen 1x3 {} sharpened_images/{}
What it does is pipe a list of files in a folder to GraphicsMagick for sharpening and outputting to a sharpened_images directory. The {} in the command is where the filenames go in the sharpening command.
This worked fine in Ubuntu GNOME 12.10 but stopped doing so after I upgraded to the next version. A look on the web set me to running the following command:
parallel --version
That produced output that included the following line:
WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.
Rerunning the original command with the --gnu option worked, but there was a more permanent solution than using something like this:
ls *.* | parallel --gnu gm convert -sharpen 1x3 {} sharpened_images/{}
That was editing /etc/parallel/config with root privileges to delete the --tollef option from there. With that completed, all was as it should again, and it makes me wonder why the change was made in the first place. Perhaps because of it, there even is a discussion about the possibility of removing the --tollef option altogether, since it raises more questions than it answers.