Renaming multiple files in Linux
19th August 2012The Linux and UNIX command mv has a number of limitations, such as not overwriting destination files and not renaming multiple files using wildcards. The only solution to the first that I can find is one that involves combining the cp and rm commands. For the second, there's another command: rename. Here's an example like what I used recently:
rename s/fedora/fedora2/ fedora.*
The first argument in the above command is a regular expression much like what Perl is famous for implementing; in fact, it is Perl-compatible ones (PCRE) that are used. The s before the first slash stands for substitute, with fedora being the string that needs to be replaced and fedora2 being what replaces it. The third command is the file name glob that you want to use, fedora.* in this case. Therefore, all files in a directory named fedora will be renamed fedora2 regardless of the file type. The same sort of operation can be performed for all files with the same extension when it needs to be changed, htm to html, for instance. Of course, there are other uses, but these are handy ones to know.
Removing VMware Player from Linux Mint Debian Edition
4th August 2012A whole slew of updates has appeared for my Linux Mint Debian Edition PC. However, to instate them, I needed to remove VMware Player and this is the command to do so:
sudo vmware-installer -u vmware-player
It worked in my case, and my system updates are in progress as I write this. The same command should work for other Linux distros where VMware Player was installed using the *.bundle installer. VMware Player remains in place on my main PC though, so I am not ditching it just yet, even if I have to be careful when running it on Linux Mint 13 so as not to freeze the system on myself.
Listing hardware information for Linux systems
3rd August 2012Curiosity about the graphics card on my backup PC caused me to look for ways of getting this information without opening up the machine or searching for a manual. In the end, a solitary command did the job:
sudo lshw
If you are running it as root, the sudo piece can be dropped, but the result is the same. As it happened, it provided me with the information that I needed.
WARNING: Engine XPORT does not support SORTEDBY operations. SORTEDBY information cannot be copied.
24th July 2012When recently creating a transport file using PROC COPY and the XPORT library engine, I found the above message in the log. The code used was similar to the following:
libname tran xport "c:\temp\tran.xpt";
proc copy in=data out=tran;
run;
When I went seeking out the cause on the web, I discovered this SAS Note that dates from before the release of SAS 6.12, putting the issue at more than ten years old. My take on its continuing existence is that we still to use a transport file format that was introduced in SAS 5.x for the sake of interoperability, both between SAS versions and across alternatives to the platform.
The SORTEDBY flag in a dataset header holds the keys used to sort the data, and it isn't being copied into the XPORT transport files, hence the warning. To get rid of it, you need to remove the information manually in the data step using the SORTEDBY option on the DATA statement or using PROC DATASETS, which avoids rewriting the entire data set.
First up is the data step option:
data test(sortedby=_null_);
set sashelp.class;
run;Then, there's PROC DATASETS:
proc datasets;
modify test(sortedby=_null_);
run;
quit;It might seem counterproductive to exclude the information, but it makes no sense to keep what's being lost anyway. So long as the actual sort order is unchanged, and I believe that the code that that below will not alter it, we can live with its documentation in a specification until transport files created using PROC CPORT are as portable as those from PROC COPY.
Installing VMware Player 4.04 on Linux Mint 13
15th July 2012Curiosity about the Release Preview of Windows 8 saw me running into bother when trying to see what it's like in a VirtualBox VM. While doing some investigations on the web, I saw VMware Player being suggested as an alternative. Before discovering VirtualBox, I did have a licence for VMware Workstation and was interested in seeing what Player would have to offer. The, it was limited to running virtual machines that were created using Workstation. Now, it can create and manage them itself and without any need to pay for the tool either. Registration on VMware's website is a must for downloading it, though, but that's no monetary cost.
Once I had downloaded Player from the website, I needed to install it on my machine. There are Linux and Windows versions; it was the former that I needed, and there are 32-bit and 64-bit variants, so you need to know what your system is running. With the file downloaded, you need to set it as executable and the following command should do the trick once you are in the right directory:
chmod +x VMware-Player-4.0.4-744019.i386.bundle
Then, it needs execution as a superuser. With sudo access for my user account, it was a matter of issuing the following command and working through the installation screens to instate the Player software on the system:
sudo ./VMware-Player-4.0.4-744019.i386.bundle
Those screens proved easy for me to follow, so life would have been good if that were all that was needed to get Player working on my PC. Having Linux Mint 13 means that the kernel is of the 3,2 stock and that means using a patch to finish off the Player installation because the required VMware kernel modules seem to silently fail to compile during the installation process. This only manifests itself when you attempt to start VMware Player afterwards to find a module installation screen appear. That wouldn't be an issue of itself were it not for the compilation failure of the vmnet module and subsequent inability to start VMware services on the machine. There is a prompt to peer into the log file for the operation, and that is a little uninformative for the non-specialist.
Rummaging around the web brought me to the requisite patch, and it works for Player 4.0.3 and Workstation 8.0.2 by default. Doing some tweaking allowed me to make it work for Player 4.04 too. My first step was to extract the contents of the tarball to /tmp where I could edit patch-modules_3.2.0.sh. Line 8 was changed to the following:
plreqver=4.0.4
With the amendment saved, it was time to execute the shell script as a superuser, having made it executable beforehand. This can be accomplished using the following command:
chmod +x patch-modules_3.2.0.sh && sudo ./patch-modules_3.2.0.sh
With that completed successfully, VMware Player ran as it should. An installation of Windows 8 into a new VM ran very smoothly, and I was impressed with the performance and responsiveness of the operating system within a Player VM. There are a few caveats, though. First, it doesn't run at all well with VMware Tools, so it's best to leave them uninstalled since it doesn't seem to need them either; it was possible to set the resolution to the same as my screen and use the CTRL+ALT+ENTER shortcut to drop in and out of full screen mode anyway. Second, the unattended Windows installation wasn't the way forward for setting up the VM, but it was no big deal to have that experiment thwarted. The feature remains an interesting one, though.
With Windows 8 running so well in Player, I was reminded of the sluggish nature of my Windows 7 VM and an issue with a Fedora 17 one too. The result was that I migrated the Windows 7 VM from VirtualBox to VMware, and all is so much more responsive. Getting it there took not a little tinkering, so that's a story for another entry. Based on my experiences so far, I reckon that VMware Player will remain useful to me for a little while yet. Resolving the installation difficulty was worth that extra effort.
ERROR: Ambiguous reference, column xx is in more than one table.
5th May 2012Sometimes, SAS messages are not all that they seem, and a number of them are issued from PROC SQL when something goes awry with your code. In fact, I got a message like the above when ordering the results of the join using a variable that didn't exist in either of the datasets that were joined. This type of thing has been around for a while (I have been using SAS since version 6.11, and it was there then) and it amazes me that we haven't seen a better message in more recent versions of SAS; it was SAS 9.2 where I saw it most recently.
proc sql noprint;
select a.yy, a.yyy, b.zz
from a left join b
on a.yy=b.yy
order by xx;
quit;Changing from to Nvidia Graphics Drivers on Linux Mint Debian Edition 64-bit
22nd April 2012One way of doing this is to go to the Nvidia website and download the latest file from the relevant page on there. Then, the next stage is to restart your PC and choose rescue mode instead of the more usual graphical option. This drops you onto a command shell that is requesting your root password. Once this is done, you can move onto the next stage of the exercise. Migrate to the directory where the *.run file is located and issue a command similar to the following:
bash NVIDIA-Linux-x86_64-295.40.run
The above was the latest file available at the time of writing, which may have changed by the time that you read this. If the executable asks to modify your X configuration file, I believe that the best course is to let it do that. Editing it yourself or running nvidia-xconfig are alternative approaches if you so prefer.
Since proprietary Nvidia drivers are included in the repositories for Linux Mint Debian Edition, that may be a better course of action since you will get updates through normal system update channels. Then, the course of action is to start by issuing the following installation commands:
sudo apt-get install module-assistant
sudo apt-get install nvidia-kernel-common
sudo apt-get install nvidia-glx
sudo apt-get install kernel-source-NVIDIA
sudo apt-get install nvidia-xconfig
Once those have completed, issuing the following in turn will complete the job ahead of a reboot:
sudo m-a a-i nvidia
sudo modprobe nvidia
sudo nvidia-xconfig
If you reboot before running the above like I did, you will get a black screen with a flashing cursor instead of a full desktop because X failed to load. Then, the remedy is to reboot the machine and choose the rescue mode option, provide the root password and issue the three commands (at this point, the sudo prefix can be dropped because it's unneeded) then. Another reboot will see order restored and the new driver in place. Running the following at that point will do a check on things, as will be the general appearance of everything:
glxinfo | grep render
Synchronising package selections between Linux Mint and Linux Mint Debian Edition
18th April 2012To generate the package list on the GNOME version of Linux Mint, I used the Backup Tool. It simply was a matter of using the Backup Software Selection button and telling it where to put the file that it generates. Alternatively, dpkg can be used from the command line like this:
sudo dpkg --get-selections > /backup/installed-software.txt
After transferring the file to the machine with Linux Mint Debian Edition, I tried using the Backup Tool on there too. However, using the Restore Software Selection button and loading the required only produced an irrecoverable error. Therefore, I set to looking around the web and found a command line approach that did the job for me.
The first step is to load the software selection using dpkg by issuing this command (it didn't matter that the file wasn't made using the dpkg command, though I suspect that's what the Linux Mint Backup Tool was doing that behind the scenes):
sudo dpkg --set-selections < /backup/installed-software.txt
Then, I started dselect and chose the installation option from the menu that appeared. The first time around, it fell over but trying again was enough to complete the job. Packages available to the vanilla variant of Linux Mint but not found in the LMDE repositories were overlooked as I had hoped, and installation of the extra packages had no impact on system stability either.
sudo dselect
Apparently, there is an alternative to using dselect that is based on the much used apt-get command, but I didn't make use of it so cannot say more:
sudo apt-get dselect-upgrade
All that I can say is that the dpkg/dselect combination did what I wanted, so I'll keep them in mind if ever need to synchronise software selections between two Debian-based distributions in the future again. While the standard edition of Linux Mint may be based on Ubuntu rather than Debian, Ubuntu is itself based on Debian. Thus, the description holds here.
Uninstalling VirtualBox Guest Additions on a Linux Guest OS
8th April 2012Within the last few days, I updated my Linux Mint Debian Edition virtual machine installation to Update 4. Between not following the instructions so closely and problems with the update server, a re-installation preceded the update itself. When all was done, no desktop environment appeared, and issuing the startx command revealed that it was one of the VirtualBox drivers that was the cause of the problem. With my being unable to see any files on the VirtualBox virtual CD, something else needed doing and the executing following command (replacing [VboxAddonsFolder] with VBoxGuestAdditions-4.1.12 in my case, but it is different for each VirtualBox version) resolved the situation:
/opt/[VboxAddonsFolder]/uninstall.sh
When it was complete, a scrambled desktop began to appear, so a reboot was to set things to rights. Then, I could set to looking at what Update 4 had brought to Linux Mint Debian Edition.
Calculating geometric means using SAS
19th March 2012Recently, I needed to calculate geometric means after a break of a number of years and ended racking a weary brain until it was brought back to me. In order that I am not in the same situation again, I am recording it here and sharing it is always good too.
The first step is to take the natural log (to base e or the approximated irrational value of the mathematical constant, 2.718281828) of the actual values in the data. Since you cannot have a log of zero, the solution is either to exclude those values or substitute a small value that will not affect the overall result, as is done in the data step below. In SAS, the log function uses the number e as its base, so you need to use the log10 equivalent when base 10 logs are needed.
data temp;
set temp;
if result=0 then _result=0.000001;
else _result=result;
ln_result=log(_result);
run;Next, the mean of the log values is determined, and you can use any method of doing that so long as it gives the correct values. PROC MEANS is used here but PROC SUMMARY (identical to MEANS except it defaults to data set creation while that generates output by default; for that reason, we need to use the NOPRINT option here), PROC UNIVARIATE or even the MEAN function in PROC SQL.
proc means data=temp noprint;
output out=mean mean=mean;
var ln_result;
run;With the mean of the log values obtained, we have to take the exponential of the obtained value(s) using the EXP function. This returns values of the same magnitude as in the original data, using the formula emean.
data gmean;
set mean;
gmean=exp(mean);
run;