Technology Tales

ERROR: This range is repeated, or values overlap: - .

This is another posting in an occasional series on SAS error and warning messages that aren't as clear as they'd need to be. What produced the message was my creation of a control data set that I then wished to use to create a data-driven (in)format. It was the PROC FORMAT step that issued the message, and I got no (in)format created. However, there were no duplicate entries in the control data set as the message suggested to me, so a little more investigation was needed.

What that revealed was that there might be one variable missing from the data set that I needed to have. The SAS documentation has defined FMTNAME, START and LABEL as compulsory variables, with each of them containing the following: format name, initial value and displayed value. My intention was this: to create a numeric code variable for one containing character strings using my data-driven format, with then numbers specified within a character variable as it should be. What was missing then was TYPE.

This variable can be one of the following values: C for character formats, I for numeric informats, J for character informats, N for numeric formats and P for picture formats. Due to it being a conversion from character values to numeric ones, I set the values of TYPE to I and used an input function to do the required operations. The code for successfully creating the informat is below:

proc sql noprint;
    create table tpts as
        select distinct "_vstpt" as fmtname,
                        "I" as type,
                        vstpt as start,
                        vstpt as end,
                        strip(put(vstptnum,best.)) as label
            from test
                where not missing(vstptnum);
quit;

proc format library=work cntlin=tpts;
run;
quit;

Though I didn't need to do it, I added an END variable too for the sake of completeness. In this case, the range is such that its start and end are the same and there are cases where that will not be the case, though I am not dwelling on those.

Command line file comparison in Windows

While UNIX and Linux both have the diff command for comparing the contents of text files, the Windows counterpart was unknown to me until recently. Its name is fc, and it looks as if the f is for file and c is for comparison, though I cannot confirm that as of now. The usage of that command is not dissimilar to the way that things work with diff. Here is an example command:

fc file1.txt file2.txt > file3.txt

This compares file1.txt with file2.txt and sends the output to file3.txt. Any differences between the two files being compared appear to be more clearly labelled than in the diff output's < and > labels. That verbosity could have its uses, but the existence of the fc command is stopping envious glances at the diff one for now, just as findstr is doing the same in comparison with grep.

Renaming multiple files in Linux

The Linux and UNIX command mv has a number of limitations, such as not overwriting destination files and not renaming multiple files using wildcards. The only solution to the first that I can find is one that involves combining the cp and rm commands. For the second, there's another command: rename. Here's an example like what I used recently:

rename s/fedora/fedora2/ fedora.*

The first argument in the above command is a regular expression much like what Perl is famous for implementing; in fact, it is Perl-compatible ones (PCRE) that are used. The s before the first slash stands for substitute, with fedora being the string that needs to be replaced and fedora2 being what replaces it. The third command is the file name glob that you want to use, fedora.* in this case. Therefore, all files in a directory named fedora will be renamed fedora2 regardless of the file type. The same sort of operation can be performed for all files with the same extension when it needs to be changed, htm to html, for instance. Of course, there are other uses, but these are handy ones to know.

Removing VMware Player from Linux Mint Debian Edition

A whole slew of updates has appeared for my Linux Mint Debian Edition PC. However, to instate them, I needed to remove VMware Player and this is the command to do so:

sudo vmware-installer -u vmware-player

It worked in my case, and my system updates are in progress as I write this. The same command should work for other Linux distros where VMware Player was installed using the *.bundle installer. VMware Player remains in place on my main PC though, so I am not ditching it just yet, even if I have to be careful when running it on Linux Mint 13 so as not to freeze the system on myself.

Listing hardware information for Linux systems

Curiosity about the graphics card on my backup PC caused me to look for ways of getting this information without opening up the machine or searching for a manual. In the end, a solitary command did the job:

sudo lshw

If you are running it as root, the sudo piece can be dropped, but the result is the same. As it happened, it provided me with the information that I needed.

WARNING: Engine XPORT does not support SORTEDBY operations. SORTEDBY information cannot be copied.

When recently creating a transport file using PROC COPY and the XPORT library engine, I found the above message in the log. The code used was similar to the following:

libname tran xport "c:\temp\tran.xpt";

proc copy in=data out=tran;
run;

When I went seeking out the cause on the web, I discovered this SAS Note that dates from before the release of SAS 6.12, putting the issue at more than ten years old. My take on its continuing existence is that we still to use a transport file format that was introduced in SAS 5.x for the sake of interoperability, both between SAS versions and across alternatives to the platform.

The SORTEDBY flag in a dataset header holds the keys used to sort the data, and it isn't being copied into the XPORT transport files, hence the warning. To get rid of it, you need to remove the information manually in the data step using the SORTEDBY option on the DATA statement or using PROC DATASETS, which avoids rewriting the entire data set.

First up is the data step option:

data test(sortedby=_null_);
    set sashelp.class;
run;

Then, there's PROC DATASETS:

proc datasets;
    modify test(sortedby=_null_);
run;
quit;

It might seem counterproductive to exclude the information, but it makes no sense to keep what's being lost anyway. So long as the actual sort order is unchanged, and I believe that the code that that below will not alter it, we can live with its documentation in a specification until transport files created using PROC CPORT are as portable as those from PROC COPY.

Installing VMware Player 4.04 on Linux Mint 13

Curiosity about the Release Preview of Windows 8 saw me running into bother when trying to see what it's like in a VirtualBox VM. While doing some investigations on the web, I saw VMware Player being suggested as an alternative. Before discovering VirtualBox, I did have a licence for VMware Workstation and was interested in seeing what Player would have to offer. The, it was limited to running virtual machines that were created using Workstation. Now, it can create and manage them itself and without any need to pay for the tool either. Registration on VMware's website is a must for downloading it, though, but that's no monetary cost.

Once I had downloaded Player from the website, I needed to install it on my machine. There are Linux and Windows versions; it was the former that I needed, and there are 32-bit and 64-bit variants, so you need to know what your system is running. With the file downloaded, you need to set it as executable and the following command should do the trick once you are in the right directory:

chmod +x VMware-Player-4.0.4-744019.i386.bundle

Then, it needs execution as a superuser. With sudo access for my user account, it was a matter of issuing the following command and working through the installation screens to instate the Player software on the system:

sudo ./VMware-Player-4.0.4-744019.i386.bundle

Those screens proved easy for me to follow, so life would have been good if that were all that was needed to get Player working on my PC. Having Linux Mint 13 means that the kernel is of the 3,2 stock and that means using a patch to finish off the Player installation because the required VMware kernel modules seem to silently fail to compile during the installation process. This only manifests itself when you attempt to start VMware Player afterwards to find a module installation screen appear. That wouldn't be an issue of itself were it not for the compilation failure of the vmnet module and subsequent inability to start VMware services on the machine. There is a prompt to peer into the log file for the operation, and that is a little uninformative for the non-specialist.

Rummaging around the web brought me to the requisite patch, and it works for Player 4.0.3 and Workstation 8.0.2 by default. Doing some tweaking allowed me to make it work for Player 4.04 too. My first step was to extract the contents of the tarball to /tmp where I could edit patch-modules_3.2.0.sh. Line 8 was changed to the following:

plreqver=4.0.4

With the amendment saved, it was time to execute the shell script as a superuser, having made it executable beforehand. This can be accomplished using the following command:

chmod +x patch-modules_3.2.0.sh && sudo ./patch-modules_3.2.0.sh

With that completed successfully, VMware Player ran as it should. An installation of Windows 8 into a new VM ran very smoothly, and I was impressed with the performance and responsiveness of the operating system within a Player VM. There are a few caveats, though. First, it doesn't run at all well with VMware Tools, so it's best to leave them uninstalled since it doesn't seem to need them either; it was possible to set the resolution to the same as my screen and use the CTRL+ALT+ENTER shortcut to drop in and out of full screen mode anyway. Second, the unattended Windows installation wasn't the way forward for setting up the VM, but it was no big deal to have that experiment thwarted. The feature remains an interesting one, though.

With Windows 8 running so well in Player, I was reminded of the sluggish nature of my Windows 7 VM and an issue with a Fedora 17 one too. The result was that I migrated the Windows 7 VM from VirtualBox to VMware, and all is so much more responsive. Getting it there took not a little tinkering, so that's a story for another entry. Based on my experiences so far, I reckon that VMware Player will remain useful to me for a little while yet. Resolving the installation difficulty was worth that extra effort.

ERROR: Ambiguous reference, column xx is in more than one table.

Sometimes, SAS messages are not all that they seem, and a number of them are issued from PROC SQL when something goes awry with your code. In fact, I got a message like the above when ordering the results of the join using a variable that didn't exist in either of the datasets that were joined. This type of thing has been around for a while (I have been using SAS since version 6.11, and it was there then) and it amazes me that we haven't seen a better message in more recent versions of SAS; it was SAS 9.2 where I saw it most recently.

proc sql noprint;
    select a.yy, a.yyy, b.zz
        from a left join b
            on a.yy=b.yy
                order by xx;
quit;

Changing from to Nvidia Graphics Drivers on Linux Mint Debian Edition 64-bit

One way of doing this is to go to the Nvidia website and download the latest file from the relevant page on there. Then, the next stage is to restart your PC and choose rescue mode instead of the more usual graphical option. This drops you onto a command shell that is requesting your root password. Once this is done, you can move onto the next stage of the exercise. Migrate to the directory where the *.run file is located and issue a command similar to the following:

bash NVIDIA-Linux-x86_64-295.40.run

The above was the latest file available at the time of writing, which may have changed by the time that you read this. If the executable asks to modify your X configuration file, I believe that the best course is to let it do that. Editing it yourself or running nvidia-xconfig are alternative approaches if you so prefer.

Since proprietary Nvidia drivers are included in the repositories for Linux Mint Debian Edition, that may be a better course of action since you will get updates through normal system update channels. Then, the course of action is to start by issuing the following installation commands:

sudo apt-get install module-assistant
sudo apt-get install nvidia-kernel-common
sudo apt-get install nvidia-glx
sudo apt-get install kernel-source-NVIDIA
sudo apt-get install nvidia-xconfig

Once those have completed, issuing the following in turn will complete the job ahead of a reboot:

sudo m-a a-i nvidia
sudo modprobe nvidia
sudo nvidia-xconfig

If you reboot before running the above like I did, you will get a black screen with a flashing cursor instead of a full desktop because X failed to load. Then, the remedy is to reboot the machine and choose the rescue mode option, provide the root password and issue the three commands (at this point, the sudo prefix can be dropped because it's unneeded) then. Another reboot will see order restored and the new driver in place. Running the following at that point will do a check on things, as will be the general appearance of everything:

glxinfo | grep render

Synchronising package selections between Linux Mint and Linux Mint Debian Edition

To generate the package list on the GNOME version of Linux Mint, I used the Backup Tool. It simply was a matter of using the Backup Software Selection button and telling it where to put the file that it generates. Alternatively, dpkg can be used from the command line like this:

sudo dpkg --get-selections > /backup/installed-software.txt

After transferring the file to the machine with Linux Mint Debian Edition, I tried using the Backup Tool on there too. However, using the Restore Software Selection button and loading the required only produced an irrecoverable error. Therefore, I set to looking around the web and found a command line approach that did the job for me.

The first step is to load the software selection using dpkg by issuing this command (it didn't matter that the file wasn't made using the dpkg command, though I suspect that's what the Linux Mint Backup Tool was doing that behind the scenes):

sudo dpkg --set-selections < /backup/installed-software.txt

Then, I started dselect and chose the installation option from the menu that appeared. The first time around, it fell over but trying again was enough to complete the job. Packages available to the vanilla variant of Linux Mint but not found in the LMDE repositories were overlooked as I had hoped, and installation of the extra packages had no impact on system stability either.

sudo dselect

Apparently, there is an alternative to using dselect that is based on the much used apt-get command, but I didn't make use of it so cannot say more:

sudo apt-get dselect-upgrade

All that I can say is that the dpkg/dselect combination did what I wanted, so I'll keep them in mind if ever need to synchronise software selections between two Debian-based distributions in the future again. While the standard edition of Linux Mint may be based on Ubuntu rather than Debian, Ubuntu is itself based on Debian. Thus, the description holds here.