Technology Tales

Append or update?

SAS can generate many types of output: plain text, XML, PDF, RTF, Excel, etc. With all of these and the SAS procedures like PROC REPORT, PROC TABULATE and so on, it might seem surprising for me to say that I have been generating output with data step PUT and FILE statements. There was, of course, a reason for this: creating text files for loading into a new database-driven software application. At one stage, I also did some data interleaving at the output stage and that’s when I discovered that the default behaviour for SAS FILE statements is to completely overwrite a file unless the MOD option was specified. Adding that switches on APPEND behaviour. The code below adds a header in one step while adding data below it in another. I know that there are slicker ways to achieve this like setting up your data as you want it or using _N_ to ensure that something only appears once but here’s another way. As per the Perl, there’s often more than one way to do something with SAS.

data _null_;
file ds_data;
put "fieldtype;datasetname;datasetlabel;datasetlayout;datasetclass;datasetstandardversion";
run;

data _null_;
set ds_ispec;
file ds_data mod;
line="datasetstandard;"||trim(memname)||";"||trim(memlabel)||";;;"||trim(memver);
put line;
run;

The irritation of a 4 GB file size limitation

I recently got myself a 500GB Western Digital My Book, an external hard drive in other words. Bizarrely, the thing is formatted using the FAT32 file system. I appreciate that backward compatibility for Windows 9x might seem desirable but using NTFS would be more understandable, particularly given that the last of the 9x line, Windows ME, is now eight years old (there cannot be anybody who still uses that, can there?). The result is that I got core dump messages from cp commands issued from the terminal on my Ubuntu system to copy files of size in excess of 4GB last night. It surprised me at first but it now seems to be a FAT32 limitation. The idea of formatting the drive as NTFS did occur to me but GParted would not do that, at least not with my current configuration. The ext3 file system is an option but I have a spare PC with Windows 2000 so that will be a step too far for now, unless I take the plunge and bring that into the Linux universe too.

Other than the 4GB irritation, the new drive works well and was picked up and supported by Ubuntu without any hassle beyond getting it out of the box, finding a place for it on my desk and plugging in a few cables. While needing judiciousness about file sizes, it played an important role while I converted a 320 GB internal WD drive from NTFS to ext3 and may yet be vital if my Windows 2000 box gets a migration to Linux. In interim, 500 GB is a lot of space and having an external drive that size is a bonus these days. That is especially the case when you consider that the 1 terabyte threshold is on the verge of getting crossed. It certainly makes DVD’s, flash drives and other multi-gigabyte media less impressive than they otherwise might appear.

A different Firefox…

On Ubuntu, I made a move to using Ubuntuzilla‘s deployment of Firefox. Because Firefox’s Gecko engine is used by other parts of Ubuntu, any Firefox updates issued by Mozilla don’t come through straight away. The idea of using Ubuntuzilla is that you get Mozilla’s latest, be it Firefox, Thunderbird or Seamonkey, without having an impact on the rest of the Linux installation; Ubuntu’s Firefox is left in place but you are now presented with the vanilla Firefox for all your web surfing needs. Visually, there’s not much change but for the built-in Firefox application fonts coming through in the new instance, a strange sight when you see Ubuntu’s more subtle alternatives everywhere else. I tried the new tack to see if picked up RealPlayer in place of Xine but that sadly has not been the case. Nevertheless, I now have 2.0.0.9 and the latest improvements this side of version 3.

About workspaces…

One of the nice things about the world of Linux and UNIX is the availability of multiple workspaces. In Window, you only ever get one and the likes of me can easily fill up that task bar. So the idea of parceling off different applications to different screens is useful from a housekeeping point of view so long as icons only appear in the task bar foe the open workspace; Ubuntu respects this but openSUSE doesn’t, a possible source of irritation.

However, a case can be made that UNIX/Linux needs workspaces more than Windows because of the multi-window interfaces of some of the software applications. The trouble with each of these sub-windows is that an entry appears in the task bar for each of this, creating a mess very quickly. And it can also be an issue working out which window closes the lot.

Examples of the above that come to my mind include GIMP, XSane and SAS. The Windows version of the latter’s DMS is confined to a single application window while the UNIX incarnation is composed of a window each for individual components like program editor, log, output, etc. Typing "bye" in the command line of the program editor is enough to dispatch the GUI. With GIMP, Ctrl+Q will close it down in any window apart from the "Tip of the Day" one that pops up when GIMP is first started. The same sort of behaviour also seems to dispatch XSane too.

Switching form one workspace to another is as easy as clicking the relevant icon in the task bar in all of the UNIX variants that I have used. Switching an application from one workspace to another has another common thread: finding the required entry in the application window menu.

In Ubuntu, I have seen other ways of working with workspaces. In the interface with visual effects turned off, hovering over the workspace icons in the task bar allows you to move from one to another with the wheel of your mouse. Moving an application between workspaces can be done as simply as dragging boxes from one task bar icon to another. Turning on the visual effects changes things, though. It might appear that the original functionality still works but that seems not to be the case: a matter for Canonical to resolve, perhaps?

The visual effects do provide other ways around this though. Keeping all your application windows minimised means that you can run through workspaces themselves with your wheel mouse. Moving applications between workspaces becomes as simple as grabbing the title bar and pulling the window left or right until it changes workspace. Be careful that you do the job fully though or you could have an application sitting astride two workspaces. It would appear that ideas from the sharing of a desktop across multiple monitors have percolated through to workspace behaviour.

Aside (regarding Ubuntu visual effects): I don’t know who came up with the idea of having windows wobble when they’re being moved around but it certainly is unusual, as is seeing what happens when you try prising a docked window from its mooring (particularly when you’re pulling it up from the bottom task bar). The sharper font display and bevelled screen furniture make more sense to me though; they certainly make a UI more appealing and modern.

A UNIX shell running on Windows

Here’s an idea that I got for a post before I spent that torrid weekend with Windows that caused me to jump ship to Linux. The idea of having a UNIX command line while still remaining in Windows did appeal to me at the time and Cygwin seems to provide an intriguing way to do this. At its most basic, it is a set of DLL’s that allow you to run standard UNIX commands in a shell like what you see below. However, it is extensible with a good number of packages that you can choose to install. NEdit is just one that gets included and I think that I spied Apache too. The standard installation is a web-based affair with your downloading only the components that you need; it’s worth trawling through the possibilities while you’re at it.

Now that I am firmly ensconced in the world of Linux, this may be one possibility that I will park, for a while anyway. After all, I now do have the full power of the UNIX command line…

BBC Radio Player and Linux

It’s been a while since I mentioned anything that might be aurally related and then this rears its head. The Xine plugin beloved by the instance of Firefox on my Ubuntu box simply refuses to play ball with the BBC’s Listen Again feature; 6Music shows are what I am trying to catch later. RealPlayer is on the system, but Firefox simply refuses to locate it. On its own, it plays live radio from the BBC and Ireland’s Today FM but, unless I need to do some digging, that’s not much use for Listen Again. It may be some hard-wiring done by Canonical as part of their packaging of Firefox: might be related to their preference for Free Software. If I can be bothered, I might replace it with the usual version to see if RealPlayer can be picked up: I do seem to remember reading somewhere that this was a possibility…

Yes, I could use one of my Windows VM’s but I have found another way courtesy of openSUSE 10.3. No, I haven’t changed distro but the reason that openSUSE has made a sudden appearance on this blog in recent times has got to do with my acquiring a copy of the latest issue of Linux Magazine. It came emblazoned with a DVD containing both 32-bit and 64-bit variants of openSUSE’s latest version and, wanting to take a look at how KDE appears these days, I knocked up a VM and installed the 32-bit variant thereupon.

Unexpectedly, that has afforded a Linux solution to my BBC Radio Player conundrum. openSUSE’s Firefox instance can find RealPlayer once you have it installed. That process involves a spot more work than if it was Free Software: you need to add an extra software repository to YaST (openSUSE’s configuration utility). The breadcrumb trail is YaST -> Software -> Software Repositories and hitting the Add button fires up a wizard that needs the following settings to set things up as needed:

Protocol: http

Server Name: download.opensuse.org

Directory on server: /distribution/10.3/repo/non-oss/suse/

Authentication: anonymous

Once the new repository was set up (I named it Non-OSS), I found the RPM and YaST took care of the rest. So, what started out as an exercise in curiosity has now found a use. Network traffic may cause playback to stutter but I have what I want without once starting up Windows. Sorting out Ubuntu may happen but it is a lesser priority and I don’t want to disrupt my computing environment in any event.

Setting up openSUSE in VMware Workstation

It should have been as straightforward as following the instructions on the openSUSE website but a bug in VMware Tools derailed things on me. The usual procedure would have you starting by selecting Install VMware Tools from the VM menu before popping into the virtual machine to do the rest. Once binutils, gcc, gcc-c++, kernel-source and make are in place, next steps should involve using YaST to install the RPM for you to run the vmware-config-tools.pl script from the terminal.

However, a bug in compat_slab.h puts a stop to any hopes of installing the vmhgfs component. That’s needed if you want to enable the shared folders feature; looking in /mnt/hgfs then would get you to any shared folders Everything else will be there but why miss out on one piece of functionality when it come in useful?

Having found a useful thread on the subject, here’s my way forward: it is as the expected procedure up to the point of installing the RPM. With VMware Tool installation on a Linux guest, you have two options: use RPM as described or use the compressed tarball. The latter seems the better course. Extract the contents into a folder and navigate to that folder. When there go into vmware-tools-distrib/lib/modules/source and extract the file vmhgfs.tar. Proceed into the resulting vmhgfs-only contained wherever you put it and perform the following edit of compat_slab.h:

Change

#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 22) || defined(VMW_KMEMCR_HAS_DTOR)

to

#if LINUX_VERSION_CODE <= KERNEL_VERSION(2, 6, 22) || defined(VMW_KMEMCR_HAS_DTOR)

After that, recreate and replace vmhgfs.tar before issuing the following command in the terminal window while in the vmware-tools-distrib directory: ./vmware-config-tools.pl (anything prefixed with "./" picks up the file from the current working directory rather than where system binaries are stored). A kernel compilation will be involved but all of the defaults should be sensible. Hopefully, all will work well after this.

Update: I am left with a number of outstanding issues that I need to resolve. Lack of internet access from the VM is one of them and a constant forgetfulness regarding the nationality of my keyboard (it’s British) might be another. In the interim, I have removed VMware tools until I can spend some time setting these to rights. Internet access has returned and British keyboard layout is being interpreted correctly in the meantime…

Importing bookmarks into Firefox

Moving from one operating system to another like I have means that a certain amount of migration is in order. I have already talked about migrating my email but there are lesser acts too. One of these is carrying across bookmarks into the new world. This should be an easy thing to achieve and, for the most part, it is. However, the Import… entry on the File menu of the main browser only brings in bookmarks from other applications. To get more flexibility, you need to open up the Bookmarks Manager window from the Bookmarks menu (Organise Bookmarks… is the entry that you need). The File menu of the Bookmarks Manager has entries named Import… and Export…; their functions should be very apparent. The former will read from a file, very useful if you do not want to disrupt what you already have. Another migration option is the potentially disruptive act of copying in an alternative bookmarks.html file into your Firefox profile folder and overwriting the one that’s already there.

Moving Emails from Outlook to Evolution

It seems a little strange to my eyes, but Evolution cannot import Outlook PST files. On one level, I see a certain amount of sense: after all, Outlook is a Windows application and Evolution remains resolutely on the Linux side of the divide. Nevertheless, it is still a pesky nuisance.

The cure is, very oddly, to import data from Outlook into Mozilla Thunderbird and pop the Thunderbird files into the Evolution mail folder. Both Evolution and Thunderbird share the same file formats, so all is hunky-dory since Evolution should just realise that they are there and bring them in.

That’s what happened for me and I have now migrated all of my old emails. Evolution’s single file import wizard is there for those times when a spot of extra persuasion is needed; the data files are those without the file extensions. As it happened, I didn’t need it.

Turning the world on its head: running VMware on Ubuntu

When Windows XP was my base operating system, I used VMware Workstation to peer into the worlds of Windows 2000, Solaris and various flavours of Linux, including Ubuntu. Now that I am using Ubuntu instead of what became a very flaky XP instance, VMware is still with me and I am using it to keep a foot in the Windows universe. In fact, I have Windows 2000 and Windows XP virtual machines available to me and they should supply my Windows needs.

A evaluation version of Workstation 6 is what I am using to power them and I must admit that I am likely to purchase a license before the evaluation period expires. Installation turned out to be a relatively simple affair, starting with my downloading a compressed tarball from the VMware website. The next steps were to decompress the tarball (Ubuntu has an excellent tool, replete with a GUI, for this) and run vmware-install.pl. I didn’t change any of the defaults and everything was set up without a bother.

In use, a few things have come to light. The first is that virtual machines must be stored on drives formatted with EXt3 or some other native Linux file system rather than on NTFS. Do the latter and you get memory errors when you try starting a virtual machine; I know that I did and that every attempt resulted in failure. After a spot of backing up files, I converted one of my SATA drives from NTFS to Ext3. For sake of safety, I also mounted it as my home directory; the instructions on Ubuntu Unleashed turned out to be invaluable for this. I moved my Windows 2000 VM over and it worked perfectly.

Next on the list was a serious of peculiar errors that cam to light when I was attempting to install Windows XP in a VM created for it. VMware was complaining about a CPU not being to run fast enough; 2 MHz was being stated for an Athlon 64 3000+ chip running at 1,58 GHz! Clearly, something was getting confused. Also, my XP installation came to a halt with a BSOD stating that a driver had gone into a loop with Framebuf fingered as the suspect. I was seeing two symptoms of the same problem and its remedy was unclear. A message on a web forum put the idea of rebooting Ubuntu into my head and that resolved the problem. I’ll be keeping an eye on it, though.

Otherwise, everything  seems to be going well with this approach and that’s an encouraging sign. It looks as my current Linux-based set up is one with which I am going to stay. This week has been an interesting one already and I have no doubt that I’ll continue to learn more as time goes on.