Adventures & experiences in contemporary technology
What you see above is a an error issued by a SAS program like what a colleague at work recently found. The following code will reproduce this so let us walk through the steps to explain a possible cause for this.
The first stage is to create a test dataset containing variables y and x, for the vertical and midpoint axes, respectively, and populating these using a CARDS statement in a data step:
data a;
input y x;
cards;
1 5
3 9
;
run;
Now, we define an axis with tick marks for particular values that will be used as the definition for the midpoint or horizontal axis of the chart:
axis1 order=(1 3);
Then, we try creating the chart using the GCHART procedure that comes with SAS/GRAPH and this is what results in the error message being issued in the program log:
proc gchart data=a;
vbar x / freq=y maxis=axis1;
run;
quit;
The cause is that the midpoint axis tick marks are no included in the data so changing these to the actual values of the x variable removes the message and allows the creation of the required chart. Thus, the AXIS1 statement needs to become the following:
axis1 order=(5 9);
Another solution is to remove the MAXIS option from the VBAR statement and let GCHART be data driven. However, if requirements do not allow this, create a shell dataset with all expected values for the midpoint axis with y set 0 since that is used for presenting frequencies as per the FREQ option in the VBAR statement.
Today, I was lured into upgrading my 2012 Google (ASUS) Nexus 7 to the final version of Android 5.0 (also known as Lollipop) by an icon in the device’s top panel. Initially, it felt as it was working OK but a certain sluggish could not be overlooked and there have been complaints about this with some questioning the sense of what Google have done. However, there would have been comments about grandfathering the device if they had not left us have the latest release of Android so there was no victory either way. We humans are fickle creatures and there is an example of exactly that in a well observed double-ended short story by the Irish writer Maura Laverty.
My impressions of how the upgrade had lumbered the tablet had me wondering about replacing the thing with either an Apple iPad Mini 2 or a Google (HTC) Nexus 9 but a much less expensive option came to mind: doing a full factory reset of the device using its recovery mode. That may sound drastic but much of what I had on there was in the cloud anyway so there was nothing to lose. So these are the instructions from Google themselves and I will leave you to use them at your own risk:
Note: If your tablet becomes unresponsive at any point during these steps, you can restart it by holding down the Power button for several seconds.
Once that was completed and the tablet restarted, the set-up routine began and took around an hour to reinstate the various apps that had been lost by the rest. Much of that was down to the time taken for re-installation rather than that taken by the actual downloads themselves over a wired broadband connection. The wait was worth it because the Nexus 7 feels more responsive again. While there times when little lags are noticeable, they are nothing next to the slowdown that I had witnessed before the rest. It might have been a better option than attempting to return to Android 4.4.4 using a factory image, which was another option that I was considering. So long as there is no deterioration in speed, the effort expended to do a reset will have been worthwhile.
It appears that PROC SGPLOT and other statistical graphics procedures create image files even if you are creating RTF or PDF files. By default, these are PNG files but there are other possibilities. When working with PC SAS , I have seen them written to the current working directory and that could clutter up your folder structure, especially if they are unwanted.
Being unable to track down a setting that controls this behaviour, I resolved to find a way around it by sending the files to the SAS work directory so they are removed when a SAS session is ended. One option is to set the session’s working directory to be the SAS work one and that can be done in SAS code without needing to use the user interface. As a result, you get some automation.
The method is implicit though in that you need to use an X statement to tell the operating system to change folder for you. Here is the line of code that I have used:
x "cd %sysfunc(pathname(work))";
The X statement passes commands to an operating system’s command line and they are enclosed in quotes. %sysfunc then is a macro command that allows certain data step functions or call routines as well as some SCL functions to be executed. An example of the latter is pathname and this resolves library or file references and it is interrogating the location of the SAS work library here so it can be passed to the operating systems cd (change directory) command for processing. This method works on Windows and UNIX so Linux should be covered too, offering a certain amount of automation since you don’t have to specify the location of the SAS work library in every session due to the folder name changing all the while.
Of course, if someone were to tell me of another way to declare the location of the generated PNG files that works with RTF and PDF ODS destinations, then I would be all ears. Even direct output without image file creation would be even better. Until then though, the above will do nicely.
Recently, I needed to put in place some code to detect the presence or absence of a variable in a dataset and I chose SAS Macro programming as the way to do what I wanted. The logic was based on a SAS sample that achieved the same result in a data step and some code that I had for detecting the presence or absence of a dataset. Mixing the two together gave me something like the following:
%macro testvar(ds=,var=);
%let dsid=%sysfunc(open(&ds,in));
%let varexist=%sysfunc(varnum(&dsid,&var));
%if &dsid > 0 %then %let rc=%sysfunc(close(&dsid));
%if &varexist gt 0 %then %put Info: Variable &var is in the &ds dataset;
%else %put Info: Variable &var is not in the &ds dataset;
%mend testvar;
%testvar(ds=dataset,var=var);
What this does is open up a dataset and look for the variable number in the dataset. In datasets, variables are numbered from left to right with 1 for the first one, 2 for the second and so on. If the variable is not in the dataset, the result is 0 so you know that it is not there. All of this is what the VARNUM SCL function within the SYSFUNC macro function does. In the example, this resolves to %sysfunc(varnum(&dsid,var)) with no quotes around the variable name like you would do in data step programming. Once you have the variable number or 0, then you can put in place some conditional logic that makes use of the information like what you see in the above simple example. Of course, that would be expanded to something more useful in real life but I hope it helps to show you the possibilities here.
More than a decade of computer upgrades and rebuilds can leave obsolete kit in your hands and the arrival of legislation controlling the dumping of electronic goods during this time can leave one wondering how anyone can dispose of them. Thankfully, I discovered that the local council refuse site only a few miles away from me accepts such things for recycling and saw me a good few times over the last summer with obsolete and non-working gadgets that has stayed with me far too long. Some were as bulky as a computer monitor and a printer but others were relatively diminutive.
Disposing of non-working and utterly obsolete equipment is an easy choice but I find this is harder when a device still works as intended and even might have a use yet. When you realise that computer motherboards still come with PS/2, floppy and IDE ports, things get trickier. My Gigabyte Z87-HD3 mainboard just has one PS/2 when predecessors would have had two and the same applies to IDE sockets and there still is a floppy drive socket on there too, a surprising sight for anyone used to thinking that such things are utterly outmoded these days. So, PC technology isn’t relinquishing backwards compatibility just yet since that mainboard is part of a system with an Intel Core i5-4670K CPU and 24 GB of RAM on there.
Even with that presence of an IDE port, I was not tempted to use leftover 10 GB and 20GB hard drives that I have had for just over a decade. Ten years ago, that sort of capacity would been respectable were it not for our voracious appetite for data storage thanks to photography, video and music. Apart from the size constraints, the speed of those drives cannot compare well with what we have today either and I quickly saw that when I replaced a Samsung 160 HD of a similar age with a Samsung SSD.
The result of this line of thought was that I was minded to recycle the drives so I started to think about wiping and Linux has a good tool for this in the form of the dd command. It can overwrite data on the disks so as to render the information virtually irretrievable. Also, Linux has a number of dummy devices that can supply junk data for overwriting purposes. They are like /dev/null which is used to suppress the issuing of output to the command. The first is /dev/zero which supplies octal zeros and I have used this. However, there also is /dev/random and /dev/urandom for those wanting a more random element to the overwriting.
To overwrite data on a disk with zeroes while having feedback on progress, the following command achieves the required result:
sudo dd if=/dev/zero | pv | sudo dd of=/dev/sdd bs=16M
The whole operation needs to be executed with root privileges and the if parameter of dd specifies the input data and this is sent to a pv command that shows a progress bar that dd would not produce by itself while sending the output on to another dd command with the disk to be overwritten specified using the of parameter. The bs parameter in that second dd command specifies the block size for the disk writing job. Unfortunately, pv is not installed by default so you need to add it yourself. On a Debian, Ubuntu or Linux Mint system, the command is the following:
sudo apt-get install pv
That pv sandwich also is invaluable for those times when dd is needed to copy partitions between different physical or virtual (in a virtual machine) disks. Without it, you might wonder what exactly is happening in the silence and that especially is concerning when you are retrying an operation that failed previously and it takes a while to complete each time.
This is another posting in an occasional series on SAS error and warning messages that aren’t as clear as they’d need to be. What produced the message was my creation of a control data set that I then wished to use to create a data-driven (in)format. It was the PROC FORMAT step that issued the message and I got no (in)format created. However, there were no duplicate entries in the control data set as the message suggested to me so a little more investigation was needed.
What that revealed was that there might be one variable missing from the data set that I needed to have. The SAS documentation has FMTNAME, START and LABEL as compulsory variables with they containing the following: format name, initial value and displayed value. My intention was to create a numeric code variable for one containing character strings using my data-driven format with then numbers specified within a character variable as it should be. What was missing then was TYPE.
This variable can be one of the following values: C for character formats, I for numeric informats, J for character informats, N for numeric formats and P for picture formats. Due to it being a conversion from character values to numeric ones, I set the values of TYPE to I and used an input function to do the required operations. The code for successfully creating the informat is below:
proc sql noprint;
create table tpts as
select distinct "_vstpt" as fmtname,
"I" as type,
vstpt as start,
vstpt as end,
strip(put(vstptnum,best.)) as label
from test
where not missing(vstptnum);
quit;
proc format library=work cntlin=tpts;
run;
quit;
Though I didn’t need to do it, I added an END variable too for sake of completeness. In this case, the range is such that its start and end are the same and there are cases where that will not be the case though I am not dwelling on those.
When recently creating a transport file using PROC COPY and the XPORT library engine, I found the above message in the log. The code used was similar to the following:
libname tran xport "c:\temp\tran.xpt";
proc copy in=data out=tran;
run;
When I went seeking out the cause on the web, I discovered this SAS Note that dates from before the release of SAS 6.12, putting the issue at more than ten years old. My take on its continuing existence is that we still to use a transport file format that was introduced in SAS 5.x for sake of interoperability, both between SAS versions and across alternatives to the platform.
The SORTEDBY flag in a dataset header holds the keys used to sort the data and it isn’t being copied into the XPORT transport files, hence the warning. To get rid of it, you need to remove the information manually in data step using the SORTEDBY option on the DATA statement or using PROC DATASETS, which avoids rewriting the entire data set.
First up is the data step option:
data test(sortedby=_null_);
set sashelp.class;
run;
Then, there’s PROC DATASETS:
proc datasets;
modify test(sortedby=_null_);
run;
quit;
It might seem counterproductive to exclude the information but it makes no sense to keep what’s being lost anyway. So long as the actual sort order is unchanged, and I believe that the code that that below will not alter it, we can live with its documentation in a specification until transport files created using PROC CPORT are as portable as those from PROC COPY.
Sometimes, SAS messages are not all that they seem and a number of them are issued from PROC SQL when something goes awry with your code. In fact, I got a message like the above when ordering the results of the join using a variable that didn’t exist in either of the datasets that were joined. This type of thing has been around for a while (I have been using SAS since version 6.11 and it was there then) and it amazes me that we haven’t seen a better message in more recent versions of SAS; it was SAS 9.2 where I saw it most recently.
proc sql noprint;
select a.yy, a.yyy, b.zz
from a left join b
on a.yy=b.yy
order by xx;
quit;
Recently, I found myself with a plot to produce but there were no data to be presented so a placeholder output is needed. For a lisitng or a table, this is a matter of detecting if there are observations to be listed or summarised and then issuing a placeholder lisitng using PROC REPORT if there are no data available. Using SAS/GRAPH, something similar can be acheived using one of its curiosities.
In the case of SAS/GRAPH, PROC GSLIDE looks like the tool to user for the same purpose. The procedure does get covered as part of a SAS Institute SAS/GRAPH training course but they tend to gloss over it. After all, there is little reason to go creating presentations in SAS when PowerPoint and its kind offer far more functionality. However, it would make an interesting tale to tell how GSLIDE became part of SAS/GRAPH in the first place. Its existence makes me wonder if it pre-exists the main slideshow production tools that we use today.
The code that uses PROC GSLIDE to create a placeholder graphic is as follows (detection of the number of observations in a SAS dataset is another entry on here):
proc gslide;
note height=10;
note j=center "No data are available";
run;
quit;
PROC GSLIDE is one of those run group procedures in SAS so a QUIT statement is needed to close it. The NOTE statements specify the text to be added to the graphic. The first of these creates a blank line of the required height for placing the main text in the middle of the graphic. It is the second one that adds the centred text that tells users of the generated output what has happened.
Recently, I needed to create a waterfall plot couldn’t use PROC SGPLOT since it was incompatible with publishing macros that use PROC GREPLAY on the platform that I was using; SGPLOT doesn’t generate plots in SAS catalogs but directly creates graphics files instead. Therefore, I decided that PROC GCHART needed to be given a go and it delivered what was needed .
The first step is to get the data into the required sort order:
proc sort data=temp;
by descending result;
run;
Then, it is time to add an ID variable for use in the plot’s X-axis (or midpoint axis in PROC GCHART) using an implied value retention to ensure that every record in the dataset had a unique identifier:
data temp;
set temp;
id+1;
run;
After that, axes have to be set up as needed. For instance, the X-axis (the axis2 statement below) needs to be just a line with no labels or tick marks on there and the Y-axis was fully set up with these, turning the label from vertical to horizontal as needed with the ANGLE option controlling the overall angle of the word(s) and the ROTATE option dealing with the letters, and a range declaration using the ORDER option.
axis1 label=none major=none minor=none value=none;
axis2 label=(rotate=0 angle=90 "Result") order=(-50 to 80 by 10);
With the axis statements declared, the GCHART procedure can be defined. Of this, the VBAR statement is the engine of the plot creation with the ID variable used for the midpoint axis and the result variable used as the summary variable for the Y-axis. The DISCRETE keyword is needed to produce a bar for every value of the ID variable or GCHART will bundle them by default. Next, references for the above axis statements (MAXIS option for midpoint axis and AXIS option for Y-axis) are added and the plot definition is complete. One thing that has to be remembered is that GCHART uses run group processing so a QUIT statement is needed at the end to close it at execution time. This feature has its uses and appears in other procedures too though SAS procedures generally are concluded by a RUN statement.
proc gchart data=temp;
vbar id / sumvar=result discrete axis=axis2 maxis=axis1;
run;
quit;