Technology Tales

Numeric for loops in Korn shell scripting: from ksh88 to ksh93

The time-honoured syntax for a for loop in a UNIX script is what you see below, and that is what works with the default shell in Sun's Solaris UNIX operating system, ksh88.

for i in 1 2 3 4 5 6 7 8 9 10
do
    if [[ -d dir$i ]]
    then
        :
    else
        mkdir dir$i
    fi
done

However, there is a much nicer syntax supported since the advent of ksh93. It follows C language conventions found in all sorts of places like Java, Perl, PHP and so on. Here is an example:

for (( i=1; i<11; i++ ))
do
    if [[ -d dir$i ]]
    then
        :
    else
        mkdir dir$i
    fi
done

New FileZilla

First, I must admit that the release of FileZilla 3 passed me by until recently. From the user interface point of view, the changes don't look too radical, but it is now cross-platform, a bonus for Linux and Mac users. It can also co-exist with FileZilla 2 for those Windows users needing features from that offering that aren't yet available in FileZilla 3. That does pose the question: why upgrade when that which you have works just as well? It is just as well that transferring settings is as easy as importing the FileZilla 2 settings into its successor is as easy as importing an XML file: in version 3, go to Edit > Import... on the menus and pick up the FileZilla.xml file from the installation directory for version 2. Though you might get some warnings and I certainly did, the FTP sites that I had set up already came over intact.

Running SQL scripts with MySQL

Here's another of those little things that you forget if you aren't using them every day: running MySQL scripts using the Windows command line. Yes, you can also run SQL commands interactively, but there's a certain convenience about scripts. I am putting an example here so that it can be found again easily:

mysql -u username -p databasename < script.sql

Though I wouldn't be at all surprised if the same line worked under Linux and UNIX, I haven't needed to give it a try.

WordPress database error: [Can’t create table … (errno: 121)]

When I was trying to upgrade one of my test blogs to WordPress 2.3 RC1, I got error messages like the above littering the screen during the installation. The PHP functions mysql_noerr (or mysqli_noerr) and mysql_error (or mysqli_error) seem to have been busily at work. These messages told me that the upgrade hadn't worked, so I went off googling as usual and perusing the MySQL tomes in my possession. Not for the first time, the web yielded nothing but dross and, in the end, I tried deleting the relevant database and starting from scratch again. That resolved the problem.

The reason for database deletion sorting things out is that MySQL got confused when there was a mismatch between what was in the file system and what its InnoDB table was saying. Now, I think that the cause of this was that I naively copied in tables using Windows Explorer. Deleting the database cleared the air and all was well once I allowed WordPress to do the needful in its time-honoured way. With another lesson learned for the future, I wish that frustration wasn't part of the learning experience too...

WARNING: The quoted string currently being processed has become more than 262 characters long…

This is a SAS error that can be seen from time to time:

WARNING: The quoted string currently being processed has become more than 262 characters long. You may have unbalanced quotation marks.

In the days before SAS version 8, this was something that needed to be immediately corrected. In these days of SAS character variables extending beyond 200 characters in length, it becomes a potential millstone around a SAS programmer's neck. If you run a piece of code like this:

data _null_;
    x="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
run;

What you get back is the warning message at the heart of the matter. While the code is legitimate and works fine, the spurious error is returned because SAS hasn't found a closing quote by the required position and the 262-character limit is a hard constraint that cannot be extended. There is another way, though: the new QUOTELENMAX option in SAS9. Setting it as follows removes the messages in most situations (yes, I did find one where it didn't play ball):

options noquotelenmax;

This does, however, beg the question as to how you check for unbalanced quotes in SAS logs these days; clearly, looking for a closing quote is an outmoded approach. Thanks to code highlighting, it is far easier to pick them out before the code gets submitted. The other question that arises is why you would cause this to happen anyway, but there are occasions where you assign the value of a macro variable to a data set one and the string is longer than the limit set by SAS. Here's some example code:

data _null_;
    length y $400;
    y=repeat("f",400);
    call symput("y",y)
run;

data _null_;
    x="&y";
run;

My own weakness is where I use PROC SQL to combine strings into a macro variable, a lazy man's method of combining all distinct values for a variable into a delimited list like this:

proc sql noprint;
    select distinct compress(string_var) into :vals separated by " " from dataset;
quit;

Of course, creating a long delimited string using the CATX (new to SAS9) function avoids the whole situation and there are other means, but there may be occasions, like the use of system macro variables, where it is unavoidable and NOQUOTELENMAX makes a much better impression when these arise.

Tidying dynamic URL’s

A few years back, I came across a very nice article discussing how you would make a dynamic URL more palatable to a search engine, and I made good use of its content for my online photo gallery. The premise was that URL's that look like that below are no help to search engines indexing a website. Though this is received wisdom in some quarters, it doesn't seem to have done much to stall the rise of WordPress as a blogging platform.

http://www.mywebsite.com/serversidescript.php?id=394

That said, WordPress does offer a friendlier URL display option too, which you can see in use on this blog; they look a little like the example URL that you see below, and the approach is equally valid for both Perl and PHP. Since I have been using the same approach for the Perl scripts powering my online phone gallery, now want to apply the same thinking to a gallery written in PHP:

http://www.mywebsite.com/serversidescript.pl/id/394

The way that both expressions work is that a web server will chop pieces from a URL until it reaches a physical file. For a query URL, the extra information after the question mark is retained in its QUERY_STRING variable, while extraneous directory path information is passed in the variable PATH_INFO. For both Perl and PHP, these are extracted from the entries in an array; for Perl, this array is called is $ENV and $_SERVER is the PHP equivalent. Thus, $ENV{QUERY_STRING} and $_SERVER{'QUERY_STRING'} traps what comes after the ? while $ENV{PATH_INFO} and $_SERVER{'PATH_INFO'} picks up the extra information following the file name (/id/394/ in the example). From there on, the usual rules apply regarding cleaning of any input but changing from one to another should be too arduous.

Perl vs. PHP: A Personal Experience

Ever since I converted it from a client-side JavaScript-powered affair, my online photo gallery has been written in Perl. There have been some challenges along the way, figuring out how to use hash tables has been one, but everything has worked as expected. However, I am now wondering if it is better to write things in PHP for the sake of consistency with the rest of the website. I had a go a rewriting the random photo page and, unless I have been missing something in the Perl world, things do seem more succinct with PHP. For instance, actions that formerly involved several lines of code can now be achieved in one. Reading the contents of a file into an array and stripping HTML/XML tags from a string fall into this category, and seeing the number of lines of code halving is a striking observation. I am not going to completely abandon Perl, it's a very nice language, but I do rather suspect that there is now an increased chance of my having a website whose server-side processing needs are served entirely by PHP.

Getting one’s HTTP headers in a twist

I am in the process of further linking the content of hillwalking blog into the fabric of other parts of my website. To date, this has included adding a list of all the posts to the site map and a dossier of the latest entries to the site's welcome page. One thing that started to crop up were a series of warnings of the form:

Cannot modify header information - headers already sent by...

The cause of this was my calling a PHP script that was part of my hillwalking blog installation, and this caused Bad Behaviour to be invoked. The result was that an HTTP header was being sent to create a cookie after one already had been sent to display a web page. A spot of conditional coding and the use of an extra flag resolved the problem.

Apparently, there is another surefire way of getting the same result: whitespace before or after the <?php ... ?> tag in an included script. The cookie issue is one that I can understand, but it does seem strange that an attempt is made to send HTTP header information when the latter arises. It causes loads of questions, though...

Restrictions on SAS libraries when macro catalogs are used

When you open up a SAS macro catalogue so that its entries for use by other programs, it has a major impact on the ability to change the library reference used to access the catalogue after it has apparently been unlocked.

options mstored sasmstore=bld_v001;

Using the line above will open the catalogue for reading, but there is no way to close it to change the library reference or unassign it until the SAS session is shut down. Even this line will not do the trick:

options nomstored sasmstore='';

What it means in practice is that if you have a standard macro setting up access to a number of standard macro libraries, then that setup macro needs to check for any library references used and not try to reassign them, causing errors in the process.

SQL Developer Java error

Last night, I tried starting up Oracle's SQL Developer so that I could add a listing of my hillwalking blog posts to my website's site map with a spot of PHP scripting. However, all that I got was something like that which you see below:

I must confess that this one threw me. The solution, though challenging to find (they often are, even with the abilities of Google) was to use a batch file called sqldeveloper.bat that you can find in the [installation directory]\sqldeveloper\bin directory. It does start the thing when all else seems to fail and got me up and running again. I did get that blog post listing added to the site map after all; Having more visibility of the MySQL tables was a definite plus point.