Technology Tales

Harnessing the power of ImageMagick

Using the command line to process images might sound senseless, only for the tools offered by ImageMagick certainly prove that it has its place. I have always been wary of using bulk processing for my digital photo files (some digitised from film prints with a scanner) but I do agree that some of it is needed to free up some time for other more necessary things. With this in mind, it is encouraging to see the results from ImageMagick and I can see it making a major difference to how I maintain my online photo gallery.

For instance, making thumbnail images for the gallery certainly seems to be one of those operations where command line bulk processing comes into its own, and ImageMagick's own convert command is heaven sent for this one. For resizing images, all that's needed is the following:

convert -resize 40% input.jpg output.jpg

Add a spot of further shell scripting and even a dash of Perl and the possibilities for this sort of thing become clearer, and this is but the pinnacle of the proverbial iceberg. The -rotate switch will do what the name suggests, while there are a whole plethora of other options on tap. So long as you have Ghostscript on your system, conversion of graphics to Postscript (and Encapsulated Postscript too) and PDF files is possible with the -page option controlling the margin around the image itself in the resulting outputs. Unfortunately, portrait is the sole orientation on offer, yet a bit of judicious post-processing will turn things around. Here's a command that'll do the trick:

convert -page 792x612+72+72 input.png ps2:output.ps

For retrieving image metadata like its resolution and size, the identify command comes into play. The -verbose option invokes the output of all manner of image metadata, so using grep or egrep is perhaps advisable, especially for bulking processing with the likes of Perl. Having the ability to stream image metadata makes loading databases like MySQL less of a chore than the manual data entry that has been my way of doing things until now.

A way to combine PDF files in UNIX and Linux

My latest adventure in the world of computing has led me into the world of automated PDF generation. When my first approach didn't prove to be completely trouble-free, I decided to look at the idea of going part of the way with it and finishing off the job with the open source utility Ghostscript. It is that which got me thinking about combining bookmarked PDF files and I can say that Ghostscript is capable of producing what I need as long it doesn't generate any errors along the way. Here's the command that does the trick:

gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -sOutputFile=final.pdf source_file1.pdf source_file2.pdf

The various switches of the gs command have very useful roles with dBATCH ensuring that Ghostscript shuts down when all is done, dNOPAUSE removing any prompts that would otherwise be given, q for quiet mode, sDEVICE using Ghostscript's own PDF creation functionality and sOutputFile creates the output file, stopping Ghostscript from sending it to its default stream. All of this applies to Windows Ghostscript too, though the name of the executable is gswin32c for 32-bit Windows instead of gs.

When it comes to any debugging, it is useful to consider that Ghostscript is case-sensitive with its command line switches, something that I have seen to trip up others. I am getting initial device initialisation, so it strikes me that dropping some of the ones that reduce the number of messages might help me work out what's going on. It's a useful idea that I have yet to try.

There is also online documentation if you fancy learning more, and Linux.com has an article that considers other possible PDF combination tools as well. All in all, it's nice to have command line tools to do these sorts of things rather than having to use GUI applications all the time.

Controlling the post revision feature in WordPress 2.6

While this may seem esoteric for some, I like to be in charge of the technology that I use. So, when Automattic included post revision retention to WordPress 2.6, I had my reservations about how much it would clutter my database with things that I didn't need. Thankfully, there is a way to control the feature, but you won't find the option in the administration screens (they seem to view this as an advanced setting and so don't want to be adding clutter to the interface for the sake of something that only a few might ever use); you have to edit wp-config.php yourself to add it. Here are the lines that can be added and the effects that they have:

Code: define('WP_POST_REVISIONS','0');

Effect: turns off post revision retention

Code: define('WP_POST_REVISIONS','-1');

Effect: turns it on (the default setting)

Code: define('WP_POST_REVISIONS','2');

Effect: only retains two previous versions of a post (the number can be whatever you want, so long as it's an integer with a value more than zero).

Update (2008-07-23):

There is now a plugin from Dion Hulse that does the above for you and more.

Quickly surveying free disk space on UNIX and Linux

Keeping an eye on disk space on a Solaris server is important for me at work, while keeping the same top level overview is good for my use of Linux at home too. Luckily, there's a simple command that delivers the goods:

df -h 2>/dev/null

The df -h piece is what delivers the statistics while the 2> /dev/null rids the terminal of any error messages; ones stating that access has been denied are common and can cloud the picture.

A collection of lessons learnt about web hosting

Putting this blog back on its feet after a spot of web hosting bother caused me to learn a bit more about web hosting than I otherwise might have done. Here's a selection, and they are in no particular order:

• Store your passwords securely and where you can find them because you never know how a foul up of your own making can strike. For example, a faux pas with a configuration file is all that's needed to cause havoc for a database site such as a WordPress blog. After all, nobody's perfect and your hosting provider may not get you out of trouble as quickly as you might like.
• Get a MySQL database or equivalent as part of your package, rather than buying one separately. If your provider allows a trial period, then changing from one package to another could be cheaper and easier than if you bought a separate database and needed to jettison it because you changed from, say, a Windows package to a Linux one or vice versa.
• It might be an idea to avoid a reseller unless the service being offered is something special. Going for the sake of lower cost can be a false economy, and it might be better to cut out the middleman altogether and go direct to their provider. Being able to distinguish a reseller from a real web host would be nice, but I don't see that ever becoming a reality; it is hardly in the resellers' interests, after all.
• Should you stick with a provider that takes several days to resolve a serious outage? The previous host of this blog had a major MySQL server outage that lasted for up to three days, and seeing that was one of the factors that made me turn tail to go to a more trusted provider that I have used for a number of years. The smoothness of the account creation process might be another point worthy of consideration.
• Sluggish system support really can frustrate, especially if there is no telephone support provided and the online ticketing system seems to take forever to deliver solutions. I would advise strongly that a host who offers a helpline is a much better option than someone who doesn't. Saying all of that, I think that it's best to be patient and, when your website is offline, that might not be as easy you'd hope it to be.
• Setting up hosting or changing from one provider to another can take a number of days because of all that needs doing. So, it's best to allow for this and plan ahead. Account creation can be quick but setting up the website can take time while domain name transfer can take up to 24 hours.
• It might not take the same amount of time to set up Windows hosting as its Linux equivalent. I don't know if my experience was typical, but I have found that the same provider set up Linux hosting far quicker (within 30 minutes) than it did for a Windows-based package (several hours).
• Be careful what package you select; it can be easy to pick the wrong one, depending on how your host's sight is laid out and what they are promoting at the time.
• You can have a Perl/PHP/MySQL site working on Windows, even with IIS being used in place instead of Apache. The Linux/Apache/Perl/PHP/MySQL approach might still be better, though.
• The Windows option allows for .Net, ASP and other such Microsoft technologies to be used. I have to say that my experience and preference is for open-source technologies, so Linux is my mainstay, but learning about the other side can never hurt from a career point of view. After, I am writing this on a Windows Vista powered laptop to see how the other half lives, as much as anything else.
• Domains serviced by hosting resellers can be visible to the systems of those from whom they buy their wholesale hosting. This frustrated my initial attempts to move this blog over because I couldn't get an account set up for technologytales.com because a reseller had it already on the same system. It was only when I got the reseller to delete the account with them that things began to run more smoothly.
• If things are not going as you would like them, getting your account deleted might be easier than you think, so don't procrastinate because you think it is a hard thing to do. Of course, it goes without saying that you should back things up beforehand.

Mucking about with WINE

It was the prospect of having Photoshop Elements going on Linux that got me thinking about working with WINE. The cause of that was Elements' inability to edit, create and save files to a VMware shared folder. As it turned out, there was more to my WINE adventures than getting Elements working. Because I was in learning mode, those adventures turned out to be messy ones, with WINE getting uninstalled and reinstalled a number of times. For the last of these, I forced matters by installing from a DEB package rather than going through Ubuntu's normal channels. The openSUSE journey was a bit more orderly, and that VM option remains if I want to go experimenting more.

Along the way, I got the Windows version of Opera going as a test. When trying out WINE in former times, I never tried installing applications into it like I do now. I don't know if this was because I hadn't made an important connection or that it was not the way that things used to be. Flushed with the success of Opera, I went further and discovered that Dreamweaver 8 and Altova's XMLSpy 2007 Professional work without my breaking a sweat. Photoshop Elements was another story and one that I have told before. Apple's iTunes was another thing that I tried without any success, even with a useful guide on Wine Reviews; for some reason, I'm having trouble getting the installation to complete successfully. I think that I'll leave my tinkering at that for now, but my general impression is that WINE works well these days, even if there is the odd crash or inexplicable disappearance of an application window. The latter happened with Dreamweaver and XMLSpy and I needed to log off and back on again to clear the slate for further progress.

A second post today?

While I know what I said about a post every two days, something has entered my head that seems timely. Things seem to starting up for 2008 and my getting a swathe of post ideas is only one of them. Today, Sun has bought up MySQL, the database that stores these ruminations for posterity, and Oracle has finally got its hands on Bea, the people behind the Weblogic software with which I have had an indirect brush for a lot of 2007.

Setting up Quanta Plus to edit files on your web server

On Saturday, my hillwalking and photo gallery website suffered an outage thanks to Fasthosts, the site's hosting provider, having a security breach and deciding to change all my passwords. While I won't bore you with the details here, I had to change the password for my MySQL database from their unmemorable suggestion and hence the configuration file for the hillwalking blog. To accomplish this, I set up Quanta Plus to edit the requisite file on the server itself. That was achieved by creating a new project, setting the protocol as FTP and completing the details in the wizard, all relatively straightforward stuff. Since I have a habit of doing this from Dreamweaver, it's nice to see that an open source alternative provides the same sort of functionality.

Why I’ll be keeping Windows close to hand for a while to come

Even though I have moved to Linux, and it has been fulfilling nearly all of my home computing needs, I do and plan to continue to retain access to Windows courtesy of virtualisation technology. Thought keeping current with the world of the ever pervasive Windows is one motivation, there are others. In fact, now that Windows is more of a sideline, I may even get my hands on Vista at some point to take a further in-depth look at it, hopefully without having to suffer the consequences of my curiosity.

Talking of other reasons for hanging onto Windows, listening to music secured by DRM does come to mind. DRM is seen in a negative light by many in the open-source world, so Linux remains unencumbered by the beast. That isn't necessarily a bad thing, and the whole furore about Vista and DRM earlier this year had me wondering about a Linux future. However, I have been known to buy music from iTunes and would like to continue doing so. Though WINE might be one way to achieve this, retaining Windows seems a sounder option. That way, I am saved from having to convert my protected music files into either Ogg Vorbis or FLAC; the latter involves a lossless compression unlike the former, so the files are bigger with the additional quality that an audiophile would seek. MP3 is another option, yet there are those in the Linux world who frown upon anything patented. That makes getting MP3 support an additional task for those of us wanting it.

In my wisdom, I have succumbed to the delights of expensive web development tools like Altova's XMLSpy and Adobe's Dreamweaver. While I have found a way to get Quanta Plus to edit files on the web server directly and code hacking is my main way to improve my websites, I still will be having a bimble into Dreamweaver from time to time. I have yet to see XMLSpy's grid view replicated in the open-source world, so that should remain a key tool in my arsenal. While I haven't been looking too hard at open-source XML editors recently, there remains unexplored functionality in XMLSpy that I should really explore to see if it could be harnessed.

While I have included implicit references to this already, it needs saying that keeping Windows around also allows you to continue using familiar software. For some, this might be Microsoft Office, but OpenOffice and Evolution have usurped this in my case. Photoshop Elements is a better example for me. Digital transfers from scanners and DSLR's will stay in the world of Linux, while virtualisation allows me to process the images in whatever way I want. For now, I might just stick with the familiar before jumping ship to GIMP at some point in the future. With all that is written on Photoshop, having it there for learning new things seems a very sensible idea.

While open-source software can conceivably address every possible, there are bound to be niches that remain outside its reach. I use mapping software from Anquet when planning hillwalking excursions. It seems very much to be a Windows only offering and I have already downloaded a good amount of mapping, so Windows has to stay if I need to use this and the routes that I have plotted out before now. Another piece of software that finds its way into this bracket is my copy of SAS Learning Edition; there are times when a spot of learning at home goes a long way at work.

So, in summary, my reasons for keeping Windows around are as follows:

• Learning new things about the thing, since I am unlikely to escape its influence in the world of work
• Using iTunes to download new music and to continue to listen to what I have already
• Using and learning about industry standard web development tools like Dreamweaver and XMLSpy
• Easing the transition, by continuing to use Photoshop Elements, for example
• Using niche software like Anquet mapping

Though I suppose that many will relate to the above, Linux still has plenty to take over some of the above. In time, DRM may disappear from the music scene and not before time; accountants and shareholders may need to learn to trust customers. NVu and Quanta Plus could yet usurp Dreamweaver, and there may be an open-source alternative to XMLSpy like there is for so many other areas. The Photoshop versus GIMP choice will continue to prevent itself and all that is written about the former makes it seem silly to throw it away, however good the latter is. Even with changing over Linux equivalents of applications fulfilling standard needs, it still leaves niche applications like hillwalking mapping and that, together with the need to know what Windows might offer in the enterprise space, could be the enduring reasons for keeping it near to hand. That said, I can now go through whole days without firing up a Windows VM, a big change from how it was a few months ago. Still, I suppose that it's all too easy to stick with using one operating system at a time, which is Linux for me these days.

The irritation of a 4 GB file size limitation

Recently, I got myself a 500GB Western Digital My Book, an external hard drive in other words. Bizarrely, the thing is formatted using the FAT32 file system. While I appreciate that backward compatibility for Windows 9x might seem desirable, using NTFS would be more understandable, particularly given that the last of the 9x line, Windows ME, is now eight years old (there cannot be anybody who still uses that, can there?). The result is that I got core dump messages from cp commands issued from the terminal on my Ubuntu system to copy files of size exceeding 4GB last night. It surprised me at first, but it now seems to be a FAT32 limitation. The idea of formatting the drive as NTFS did occur to me, only for GParted not to do that, at least not with my current configuration. While the ext3 file system is an option, I have a spare PC with Windows 2000 so that will be a step too far for now, unless I take the plunge and bring that into the Linux universe too.

Other than the 4GB irritation, the new drive works well and was picked up and supported by Ubuntu without any hassle beyond getting it out of the box, finding a place for it on my desk and plugging in a few cables. While needing judiciousness about file sizes, it played an important role while I converted a 320 GB internal WD drive from NTFS to ext3 and may yet be vital if my Windows 2000 box gets a migration to Linux. In the interim, 500 GB is a lot of space, and having an external drive that size is a bonus these days. That is especially the case when you consider that the 1 terabyte threshold is on the verge of getting crossed. It certainly makes DVD's, flash drives and other multi-gigabyte media less impressive than they otherwise might appear.