Technology Tales

A reasonable requirement of an IDE

I have been having a play with NetBeans IDE Early Access for PHP and, while it has a lot to offer, one impression remains uppermost in my mind: it is so slow. I might have a project with a lot of files in it but start-up takes an age because of project checking. Other functionality such as text searching is far from speedy either. The sluggishness probably arises from this release being very early in its life cycle and it reminds me of how slow older versions of the Java IDE were, even if this is slower. For PHP development, I’ll be giving NetBeans a while to mature before taking another look at it.

On a similar note, I recently dispatched Quanta Plus from my system for sluggish start-ups and will not return to it because other alternatives such as Bluefish and Eclipse PDT fit my needs much better. I like my editors to be slick and responsive and Quanta has been around long enough for any slowness to be knocked out of it. However, I get the feeling that the extras have added bloat while I expect any additional functionality that I never use not to get in my way. It is for the latter reason that I was always able to get on with Dreamweaver and even run it on Ubuntu using the WINE library. If I really wanted a stripped out yet functional editor, Gedit would do most of what I need -- it colour codes syntax for a variety of languages for a start -- but it’s always handy to have a file system explorer window incorporated and I value any syntax checking and auto-completion as well. So, it looks as if Eclipse and Bluefish could be serving my needs for a while to come alongside so use Dreamweaver for online editing of website files.

Escaping brackets in SAS macro language

Rendering opening and closing brackets as pieces in SAS macro language programming caused me a bit of grief until I got it sorted a few months back. All of the usual suspects for macro quoting (or escaping in other computer languages) let me down: even the likes of %SUPERQ or %NRBQUOTE didn’t do the trick. The honours were left to %NRQUOTE(%(), which performed what was required very respectably indeed. The second "%" escapes the bracket for %NRQUOTE to do the rest.

Detecting file ownership in Korn shell scripts

I recently was having a play with using a shell script to do some folder creation to help me set up a system for testing and I started to hit ownership issues that caused some shell script errors. At the time, I didn’t realise that there is a test that you can perform for ownership. The "-o" in the code below kicks in the test condition and avoids the error in question.

if [[ -o $dirname ]]
then
    cd test
    for i in 1 2 3 4 5 6 7 8 9 10
    do
        if [[ -d study$i ]]
        then
            :
        else
            mkdir study$i
        fi
    done
    ls
    cd ~
fi

Previously, I shared a way to test for directory (-d operator) and file (-f operator) existence that follows the above coding convention. However, there are a plethora of others and I have made a list of them here:

It’s all useful stuff when you want to rid the command line output of errors in an above board sort of way. These are the kinds of things that often make life easier…

A throwback to the past: an appearance of MACROGEN

Recently, I was reviewing a log of a program being run by SAS 9.1.3 on a Solaris system and spotted lines like the following:

MACROGEN(MACRO1):   OPTIONS NOMPRINT NOMPRINTNEST

NOTE: PROCEDURE DISPLAY used (Total process time):
real time           0.73 seconds
cpu time            0.50 seconds

MPRINT(MACRO1):   SOURCE SOURCE2 NOTES;

The appearance of the word MACROGEN made me wonder if there was another system option that I had missed. A quick search of the SAS website threw up a support note that shed some light on the situation. Apparently, MACROGEN is the SAS v5 forbear of today’s MPRINT, MLOGIC, and SYMBOLGEN options and would seem to be obsolete in these days. Having started programming SAS in the days of version 6, I had missed out on MACROGEN and so use its replacements instead, hence my never coming across the option. Quite what it’s doing showing up in a SAS 9 log is another story: and there I was thinking that SAS 9 was the result of a full rewrite… Now, I am not so sure but at least I know what MACROGEN is if someone ever takes the time to ask me.

SAS9 SQL Constraints

With SAS 9, SAS Institute have introduced the sort sort of integrity constraints that have been bread and butter for relational database SQL programs but some SAS programmers may find them more restrictive than they might like. The main one that comes to my mind is the following:

proc sql noprint;
create table a as select a.*,b.var from a left join b on a.index=b.index;
quit;

Before SAS 9, that worked merrily with nary a comment but you now will see a warning like this:

WARNING: This CREATE TABLE statement recursively references the target table. A consequence of this is a possible data integrity problem.

In data step, the following still runs without a complaint:

data a;
merge a b(keep=index var);
by index;
run;

On the surface of it this does look inconsistent. From a database programmer’s point of view having to use different source and target datasets is no hardship but seems a little surplus to requirements for a SAS programmer trained to keep down the number of temporary datasets in an effort to reduce I/O and keep things tidy, an academic concept perhaps in these days of high processing power and large disks. Adding UNDO_POLICY=NONE to the PROC SQL line does make everything consistent again but I see this as being anathema to a database programming type. I do admit to indulging in the override for personal quick and dirty purposes but abiding by the constraint is how I do things for formal purposes like inclusion in an application.

WARNING: The quoted string currently being processed has become more than 262 characters long…

This is a SAS error that can be seen from time to time:

WARNING: The quoted string currently being processed has become more than 262 characters long. You may have unbalanced quotation marks.

In the days prior to SAS version 8, this was something that needed to be immediately corrected. In these days of SAS character variables extending beyond 200 characters in length, it becomes a potential millstone around a SAS programmer’s neck. If you run a piece of code like this:

data _null_;
x="[string with more than 262 characters (putting in an actual string wrecks the appearance of the website)]";
run;

What you get back is the warning message at the heart of the matter. The code is legitimate and works fine but the spurious error is returned because SAS hasn’t found a closing quote by the required position and the 262 character limit is a hard constraint that cannot be extended. There is another way though: the new QUOTELENMAX option in SAS9. Setting it as follows removes the messages in most situations (yes, I did find one where it didn’t play ball):

options noquotelenmax;

This does however beg the question as to how you check for unbalanced quotes in SAS logs these days; clearly, looking for a closing quote is an outmoded approach. Thanks to code highlighting, it is far easier to pick them out before the code gets submitted. The other question that arises is why you would cause this to happen anyway but there are occasions where you assign the value of a macro variable to a data set one and the string is longer than the limit set by SAS. Here’s some example code:

data _null_;
length y $400;
y=repeat("f",400);
call symput("y",y)
run;

data _null_;
x="&y";
run;

My own weakness is where I use PROC SQL to combine strings into a macro variable, a lazy man’s method of combining all distinct values for a variable into a delimited list like this:

proc sql noprint;
select distinct compress(string_var) into :vals separated by " " from dataset;
quit;

Of course, creating a long delimited string using the CATX (new to SAS9) function avoids the whole situation and there are other means but there may be occasions, like the use of system macro variables, where it is unavoidable and NOQUOTELENMAX makes a much better impression when these arise.

Getting one’s HTTP headers in a twist

I am in the process of further linking the content of hillwalking blog into the fabric of other parts of my website. To date, this has included adding a list of all the posts to the site map and a dossier of latest entries to the site’s welcome page. One things that started to crop up were a series of warnings of the form:

Cannot modify header information -- headers already sent by…

The cause of this was my calling a PHP script that was part of my hillwalking blog installation and this caused Bad Behaviour to be invoked. The result was that a HTTP header was being sent in order to create a cookie after one already had been sent to display a web page. A spot of conditional coding and the use of an extra flag resolved the problem.

Apparently, there is another surefire way of getting the same result: whitespace before or after the <?php … ?> tag in an included script. The cookie issue is one that I can understand but it does seem strange that an attempt is made to send HTTP header information when the latter arises. It causes loads of questions, though…

Restrictions on SAS libraries when macro catalogs are used

When you open up a SAS macro catalog so that its entries for use by other programs, it has a major impact on the ability to change the library reference used to access the catalog after it has apparently been unlocked.

options mstored sasmstore=bld_v001;

Using the line above will open the catalog for reading but there is no way to close it in order to change the library reference or deassign it until the SAS session is shut down. Even this line will not do the trick:

options nomstored sasmstore='';

What it means in practice is that if you have a standard macro setting up access to a number of standard macro libraries, then that setup macro needs to check for any library references used and not try to reassign them, causing errors in the process.

Using alternative editors for SAS programming

When it comes to writing SAS programs, most of use the tools that SAS gives us, be it Enterprise Guide, the Enhanced Editor or the Program Editor. While Enterprise Guide can work with UNIX SAS as the processing engine, it is very much a Windows tool and the Enhanced Editor functionality is provided through Windows-only programming (ActiveX, I seem to recall). However, that means that creature comforts are left behind you if you turn to writing SAS code using UNIX SAS; you have only got the good old fashioned Program Editor supplied by SAS itself. However, there is a trick that you can use to make life more comfortable: SAS does allow you to submit the contents of your paste buffer (or clipboard) using the command SUBMIT BUFFER=DEFAULT and this can be assigned to a function key for ease of use (I use the same key to clear the log and output screens at the same time). In the Windows, you may need to explicitly copy the code in order to do this but, in UNIX, merely highlighting a section of code with an editor like NEdit will do the trick and, given that NEdit is reasonably pleasant tool for code cutting (the ability to define its macros with a spot of scripting is a definite plus point), this makes life more comfortable again.

SAS books now on Safari

Being a Safari subscriber, I found a pleasant surprise awaiting me in this month’s email newsletter: eBooks from SAS Books are now available on Safari. Having a quick look, I found a small but useful selection. Topics like the SQL procedure, the Macro language and Enterprise Guide caught my eye but there’s more than this on offer. It’ll be interesting to see where this leads…