Tag Archive for processing

Creating waterfall plots in SAS using PROC GCHART

March 17, 2012 No comments

Recently, I needed to create a waterfall plot couldn’t use PROC SGPLOT since it was incompatible with publishing macros that use PROC GREPLAY on the platform that I was using; SGPLOT doesn’t generate plots in SAS catalogs but directly creates graphics files instead. Therefore, I decided that PROC GCHART needed to be given a go and it delivered what was needed .

The first step is to get the data into the required sort order:

proc sort data=temp;
by descending result;
run;

Then, it is time to add an ID variable for use in the plot’s X-axis (or midpoint axis in PROC GCHART) using an implied value retention to ensure that every record in the dataset had a unique identifier:

data temp;
set temp;
id+1;
run;

After that, axes have to be set up as needed. For instance, the X-axis (the axis2 statement below) needs to be just a line with no labels or tick marks on there and the Y-axis was fully set up with these, turning the label from vertical to horizontal as needed with the ANGLE option controlling the overall angle of the word(s) and the ROTATE option dealing with the letters, and a range declaration using the ORDER option.

axis1 label=none major=none minor=none value=none;
axis2 label=(rotate=0 angle=90 “Result”) order=(-50 to 80 by 10);

With the axis statements declared, the GCHART procedure can be defined. Of this, the VBAR statement is the engine of the plot creation with the ID variable used for the midpoint axis and the result variable used as the summary variable for the Y-axis. The DISCRETE keyword is needed to produce a bar for every value of the ID variable or GCHART will bundle them by default. Next, references for the above axis statements (MAXIS option for midpoint axis and AXIS option for Y-axis) are added and the plot definition is complete. One thing that has to be remembered is that GCHART uses run group processing so a QUIT statement is needed at the end to close it at execution time. This feature has its uses and appears in other procedures too though SAS procedures generally are concluded by a RUN statement.

proc gchart data=temp;
vbar id / sumvar=result discrete axis=axis2 maxis=axis1;
run;
quit;

Programming

, , , , , , , , , , , , , , , , , , ,

Using Data Step to Create a Dataset Template from a Dataset in SAS

November 23, 2010 No comments

Recently, I wanted to make sure that some temporary datasets that were being created during data processing in a dataset creation program weren’t truncating values or differed from the variable lengths in the original. It was then that a brainwave struck me: create an empty dataset shell using data step and use that set all the variable lengths for me when the new datasets were concatenated to it. The code turned out to be very simple and here is an example of how it looked:

data shell;
stop;
set example;
run;

The STOP statement, prevents the data step from reading in any of the values in the template dataset and just its header is written out to another (empty) dataset that can be used to set things up as you would want them to be. It certainly was a quick solution in my case.

Programming

, , , , , , , , , ,

A tendency for overexposure?

July 14, 2010 No comments

A recent trip to Sweden saw my Canon PowerShot G11 being put to rather more use than was expected. If I had known what might have been coming my way, I may even have eschewed the principle of lightweight packing to bring along my Pentax DSLR. Nevertheless, the little Canon did what ever was asked of it when light was plentiful.

Once thing that I have noticed in comparison with the Pentax is the Canon’s tendency to overexpose a scene. To a point, this can be explained by the former having proper spot-metering and the latter having the less specific partial metering. In fact, that might explain why a Canon EOS 10D SLR in my possession has the same tendency. Maybe it’s time to make more use of the Sekonic light meter that I have but that adds bulk that doesn’t fit in with the idea of carrying a compact camera around with you.

That leaves getting more practice with exposure corrections at processing time (I do capture all my photos in raw format). Going further, I am finding that the same consideration appears to apply to image sharpening too. It’s almost as if you need to develop a feeling for the results produced by a camera before satisfaction with any acquired photos will follow. Having decent lighting at capture time and not having muck on the sensor helps too as I have discovered with the photos made used my Pentax K10D on a recent visit to Arran and Argyll. The state of the sensor needs sorting (even if it has an anti-dust system on board) but I sometimes wonder if my judgement of lighting is what it used to be or whether my aspirations have gone too high. Maybe I need to slow down a little so as to set aside time for working on getting better results and with the right light, a quantity that should come with autumn and winter. In the meantime, I’ll stick with making the best of the British summer.

Photographic Equipment

, , , , , , , , , , , , , , , , , ,

8?

October 12, 2009 No comments

It now seems that we have a new version of Photoshop Elements from Adobe for every year unless you’re a Mac user. Version 7 convinced me to splash out and that gained me Camera Raw recognition of my Pentax K10D along with subtly enhanced image processing power that I have been putting to good use to get more pleasing results than I ever got before. What can be achieved by using levels, curves and the shadow/highlight adjustment tool for exposure correction has amazed me recently. Quick selection functionality has allowed me to treat skies differently from everything else in landscape photos, a flexible graduated filter if you like. It seems to work on Windows 7 along with Vista and XP so I plan to stick with it for a while yet. As you may have gathered from this, it would take some convincing to make me upgrade and, for me, version 8 doesn’t reach that mark. All in all, it seems that it is a way of giving Mac users a new release with added goodness after having to stay with 6 for so long; yes, there are new features like autotagging in the image organiser but they just don’t grab me. Given that they already have Aperture from Apple and Windows users seem to get more releases, it’s a wonder that any Mac user would toy with Elements anyway. Maybe, that’s Adobe’s suspicion too.

Graphics, Imaging, Software

, , , , , , , , , , , , , , , , ,

ImageMagick and Ubuntu 9.04

May 5, 2009 One comment

Using a command line tool like ImageMagick for image processing may sound a really counter-intuitive thing to do but there’s no need to do everything on a case by case interactive basis. Image resizing and format conversion come to mind here. Helper programs are used behind the scenes too with Ghostscript being used to create Postscript files, for example.

The subject of helper programs brings me to an issue that has hampered me recently. While I am aware that there are tools like F-Spot available, I am also wont to use a combination of shell scripting (BASH & KSH), Perl and ImageMagick for organising my digital photos. My preference for using Raw camera files (DNG & CRW) means that ImageMagick cannot access these without a little helper. In the case of Ubuntu, it’s UFRaw. However, Jaunty Jackalope appears to have seen UFRaw updated to a version that is incompatible with the included version of ImageMagick (6.4.5 as opposed to 3.5.2 at the time of writing). The result is that the command issued by ImageMagick to UFRaw – issue the command man ufraw-batch to see the details – is not accepted by the included version of the latter, 0.15 if you’re interested. It seems that an older release of UFRaw accepted the output device ppm16 (16-bit PPM files) but this should now be specified as ppm for the output device and 16 for the output depth. In a nutshell, where the parameter output-type did the lot, you now need both output-type and output-depth.

I thought of decoupling things by using UFRaw to create 16-bit PPM files for processing by ImageMagick but to no avail. The identify command wouldn’t return the date on which the image was taken. I even changed the type to 8-bit JPEG’s with added EXIF information but no progress was made. In the end, a mad plan came to mind: creating a VirtualBox VM running Debian. The logic was that if Debian deserves its reputation for solidity, dependencies like ImageMagick and UFRaw shouldn’t be broken and I wasn’t wrong. To make it fly though, I needed to see if I could get Guest Additions installed on Debian. Out of the box, the supported kernel version must be at least 2.6.27 and Debian’s is 2.6.26 so additional work was on the cards. First, GCC, Make and the correct kernel header files need to be installed. Once those are in place, the installation works smoothly and a restart sets the goodies in motion. To make the necessary Shared folder to be available, a command like the following was executed:

mount -t vboxfs [Shared Folder name] [mount point]

Once that deed was done and ImageMagick instated, the processing that I have been doing for new DSLR images was reinstated. Ironically, Debian’s version of ImageMagick, 6.3.7, is even older than Ubuntu’s but it works and that’s the main thing. There is an Ubuntu bug report for this on Launchpad so I hope that it gets fixed at some point in the near future. However, that may mean awaiting 9.10 or Karmic Koala so I’m glad to have the workaround in the meantime.

Graphics, Imaging, Linux, Scripting, Virtualisation

, , , , , , , , , , , , , , , , , , , , , , ,

Harnessing the power of ImageMagick

October 26, 2008 No comments

Using the command line to process images might sound senseless but the tools offered by ImageMagick certainly prove that it has its place. I have always been wary of using bulk processing for my digital photo files (some digitised from film prints with a scanner) but I do agree that some of it is needed to free up some time for other more necessary things. With this in mind, it is encouraging to see the results from ImageMagick and I can see it making a major difference to how I maintain my online photo gallery.

For instance, making thumbnail images for the gallery certainly seems to be one of those operations where command line bulk processing comes into its own and ImageMagick’s own convert command is heaven sent for this one. For resizing images, all that’s needed is the following:

convert -resize 40% input.jpg output.jpg

Add a spot of further shell scripting and even a dash of Perl and the possibilities for this sort of thing become clearer and this is but the pinnacle of the proverbial iceberg. The -rotate switch will do what the name suggests and there are a whole plethora of other options on tap. So long as you have Ghostscript on your system, conversion of graphics to Postscript (and Encapsulated Postscript too) and PDF files is possible with the -page option controlling the margin around the image itself in the resulting outputs. Unfortunately, portrait is the sole orientation on offer but a bit of judicious post processing will turn things around. Here’s a command that’ll do the trick:

convert -page 792×612+72+72 input.png ps2:output.ps

For retrieving image metadata like its resolution and size, the identify command comes into play. The -verbose option invokes the output of all manner of image metadata so using grep or egrep is perhaps advisable, especially for bulking processing with the likes of Perl. Having the ability to stream image metadata makes loading databases like MySQL less of a chore than the manual data entry that has been my way of doing things until now.

Graphics, Imaging, Linux, Software, UNIX

, , , , , , , , , , , , , , , , , ,
 

  • As is commonly the case with places like these, all the views that you find expressed on here in postings and articles are mine alone and not those of any organisation with which I have any association, through work or otherwise. With regards to any comments left on the site, I reserve the right to reject any that are inappropriate. Otherwise, whatever is said is the sole responsibility of whoever is leaving the comment.