Adventures & experiences in contemporary technology
It’s amazing what ideas Linux gives that you wouldn’t encounter that clearly in the world of Windows. One of these is output and command line history so a script can be created. In the Windows world, this would be called a batch file. Linux usefully has the history command and it does the needful for taking a snapshot like so:
history > ~/commands.sh
All of the commands stored in a terminal’s command history get stored in the commands.sh in the user’s home area. The command for doing the same thing from the Windows command line is not as obvious because it uses the doskey command that is intended for command line macro writing and execution. Usefully, it has a history option that tells it to output all the commands issued in a command line session. Unless, you create a file with them in there, there seems to be no way to store all those commands across sessions, unlike UNIX and Linux. Therefore, a command like the following is a partial solution that is more permanent than using the F7 key on your keyboard:
doskey /history > c:\commands.bat
Windows Powershell has something similar too and it even has aliases of history and even h. All Powershell scripts have file extensions of ps1 and the example below follows that scheme:
get-history > c:\commands.ps1
However, I believe that even Powershell doesn’t carry over command history between sessions though Microsoft are working on adding this useful functionality. They could co-opt Cygwin of course but that doesn’t seem to be their way of going about things.
This useful addition came in SAS 9.2 and I am amazed that it isn’t enabled by default. To accomplish that, you need to set the MINOPERATOR option unless someone has done it for you in the SAS AUTOEXEC or another configuration program. Thus, the safety first approach is to have code like the following:
options minoperator;
%macro inop(x);
%if &x in (a b c) %then %do;
%put Value is included;
%end;
%else %do;
%put Value not included;
%end;
%mend inop;
%inop(a);
Also, the default delimiter is the space, so if you need to change that, then the MINDELIMITER option needs setting. Adjusting the above code so that the delimiter now is the comma character gives us the following:
options minoperator mindelimiter=",";
%macro inop(x);
%if &x in (a,b,c) %then %do;
%put Value is included;
%end;
%else %do;
%put Value not included;
%end;
%mend inop;
%inop(a);
Without any of the above, the only approach is to have the following and that is what we had to do for SAS versions prior to 9.2:
%macro inop(x);
%if &x=a or &x=b or &x=c %then %do;
%put Value is included;
%end;
%else %do;
%put Value not included;
%end;
%mend inop;
%inop(a);
It may be clunky but it does work and remains a fallback in newer versions of SAS. Saying that, having the IN operator available makes writing SAS Macro code that little bit more swish so it’s a good thing to know.
The Linux and UNIX command mv has a number of limitations, such as not overwriting destination files and not renaming multiple files using wildcards. The only solution to the first that I can find is one that involves combining the cp and rm commands. For the second, there’s another command: rename. There is a command like it in Windows but this one is a little different in its syntax. Before saying more about that, here’s an example like what I used recently:
rename s/fedora/fedora2/ fedora.*
The first argument in the above command is a regular expression much like what Perl is famous for implementing; in fact, it is Perl-compatible ones (PCRE) that are used. The s before the first slash stands for substitute with fedora being the string that needs to be replaced and fedora being what replaces it. The third command is the file name glob that you want to use, fedora.* in this case. Therefore, all files in a directory named fedora will be renamed fedora2 regardless of the file type. The same sort of operation can be performed for all files with the same extension and it needing changing, htm to html, for instance. Of course, there are other uses but these are handy ones to know.
Sometimes, SAS messages are not all that they seem and a number of them are issued from PROC SQL when something goes awry with your code. In fact, I got a message like the above when ordering the results of the join using a variable that didn’t exist in either of the datasets that were joined. This type of thing has been around for a while (I have been using SAS since version 6.11 and it was there then) and it amazes me that we haven’t seen a better message in more recent versions of SAS; it was SAS 9.2 where I saw it most recently.
proc sql noprint;
select a.yy, a.yyy, b.zz
from a left join b
on a.yy=b.yy
order by xx;
quit;
Until the advent of PowerShell, Windows had been the poor relation when it came to working from the command line when compared with UNIX, Linux and so on. A recent bit of fiddling had me trying to run FTP from the legacy command prompt when I ran into problems with UNC address resolution (it’s unsupported by the old technology) and mapping of network drives. It turned out that my error 85 was being caused by an unavailable drive letter that the net use command didn’t reveal as being in use. Reassuringly, this wasn’t a Vista issue that I couldn’t circumvent.
During this spot of debugging, I tried running batch files in PowerShell and discovered that you cannot run them there like you would from the old command prompt. In fact, you need a line like the following:
cmd /c script.bat
In other words, you have to call cmd.exe like perl.exe, wscript.exe and cscript.exe for batch files to execute. If I had time, I might have got to exploring the use ps1 files for setting up PowerShell commandlets but that is something that needs to wait until another time. What I discovered though is that UNC addressing can be used with PowerShell without the need for drive letter mappings, not a bad development at all. While on the subject of discoveries, I discovered that the following command opens up a command prompt shell from PowerShell without any need to resort to the Start Menu:
cmd /k
Entering the exit command returns you to the PowerShell command line again and entering cmd /? reveals the available options for the command so you need never be constrained by your own knowledge or its limitations.
Recently, I had the opportunity to have a look at the Excel library engine again because I need to read Excel data into SAS. You need SAS Access for PC Files licensed for it to but it does simplify the process of getting data from spreadsheets into SAS. It all revolves around setting up a library pointing at the Excel file using the Excel engine. The result is that every worksheet in the file is treated like a SAS dataset even if there names contain characters that SAS considers invalid for dataset names. The way around that is to enclose the worksheet name in single quotes with the letter n straight after the closing quote, much in the same way as you’d read in text strings as SAS date values (’04MAR2010’d, for example). In order to make all of this more, I have added some example code below.
libname testxl excel 'c:\test.xls';
data test;
set testxl.'sheet1$'n;
run;
All of the above does apply to SAS on Windows (I have used it successfully in 9.1.3 and 9.2) but there seems to be a way of using the same type of thing on UNIX too. Again, SAS Access for PC Files is needed as well as a SAS PC Files server on an available Windows machine and it is the PCFILES engine that is specified. While I cannot say that I have had the chance to see it working in practice but seeing it described in SAS Online Documentation corrected my previous misimpressions about the UNIX variant of SAS and its ability to read in Excel or Access data. Well, you learn something new every day.
On thing that I recently had to investigate was the possibility of converting CGM vector graphics files into Postscript and from there into PDF. Having used ImageMagick for converting images before, that was an obvious option. However, that cannot process CGM files on its own and needs a delegate or helper application as well. This is the case with raw digital camera files too with UFRaw being the program chosen. For CGM images, the more obscure RALCGM is what’s needed and tracking it down is a bit of an art. The history is that it was developed at the U.K.’s Rutherford Appleton Laboratory but it seems that it was left go off into the wilderness rather than someone keeping an eye on things. With that in mind, here are the installation packages for Windows and Linux (RPM):
RALCGM is a handy command line tool that can covert from CGM to Postscript on its own without any need for ImageMagick at all. From what I have seen, fonts on graphical output may look greyer than black but it otherwise does its job well. However, considering that it is a freely available tool, one cannot complain too much. There are other packages for doing vector to raster conversion and the ones that I have seen do have GUI‘s but the freedom to look at for cost software wasn’t mine to have. The required command looks something like the following:
ralcgm -d PS -oL test.cgm test.ps
The switch -d PS uses the software’s Postscript driver and -oL specifies landscape orientation. If you want to find out more, here’s a PDF rendition of the help file that comes with the thing:
This is an error that I have started to see a lot in the last few weeks. First, it was with Piwik and latterly with WordPress.com Stats. For the record, I have never seen it on up to date systems but always with IE6 and at page unloading time. The CPU usage hits 100% before the error is produced and that has had me blaming JavaScript in error; it isn’t the cause of all ills. In fact, the cause seems to be a bug in a certain release of Adobe Flash 9 but I am of the opinion that the inclusion of certain features in a Flash movie are needed to trigger it too. I don’t have the exact details of this but WordPress.com Stats worked without fault until a recent update and that is what is making me reach the conclusion that I have. That observation is making me wonder whether we are coming to a point where Flash compatibility is something that needs to factored into the use of the said technology in a website or web application. Updating Flash will solve the problem on the client but it might be better if it wasn’t triggered on the server side either.
This piece is as much an aide de memoire for myself as anything else but putting it here seems worthwhile if it answers questions for others. The binding operators, =~ or !~, come in handy when you are framing conditional statements in Perl using Regular Expressions, for example, testing whether x =~ /\d+/ or not. The =~ variant is also used for changing strings using the s/[pattern1]/[pattern2]/ regexp construct (the “s” stands for “substitute”). What has brought this to mind is that I wanted to ensure that something was done for strings that did not contain a certain pattern and that’s where the !~ binding operator came in useful; ^~ might have come to mind for some reason but it wasn’t what I needed.
Fasthosts, the hosting provider for what you find here has, in their wisdom, decided to limit the execution time for ASP scripts to 15 seconds and 10 seconds for any others. I haven’t used Perl sufficiently in this shared hosting set up to determine how that is affected. In contrast, I can share my experiences on the PHP side and you may have noticed occasional glitches. They have also disabled the set_time_limit PHP function so you cannot easily address the matter yourself where you need to do it. You almost get the feeling that they don’t trust the abilities, actions and oversight of their users. Personally, I reckon that the ten second limit is too short and that something of the order of 20 or 30 seconds would be better. If it all gets too restrictive, I suppose that there are other providers though I think that I would avoid resellers after a previous less than glorious experience. There’s the dedicated server option too if I was feeling flush, not so likely given the economic times in which we live.