SAS functions for character and string validation
22nd January 2010There is a whole pile of SAS functions for testing text strings that hadn't come to my attention until this week. Until then, I'd have gone about using functions like INDEX and PRXMATCH functions for the same sort of ends, but it's never any load to have a few different ways of doing things and to use the right one for the job. Here's a quick list of my recent discoveries:
ANYALNUM: First position of any alphanumeric character, returns 0 if absent
ANYALPHA: First position of any alphabetic character (letter of the alphabet), returns 0 if absent
ANYCNTRL: First position of any control character, returns 0 if absent
ANYDIGIT: First position of any numeric character, returns 0 if absent
ANYFIRST: First position of any character that can be used as the start of a SAS variable name when VALIDVARNAME is set to V7, returns 0 if absent
ANYGRAPH: First position of any printable character that isn't white space, returns 0 if absent
ANYLOWER: First position of any lowercase letter, returns 0 if absent
ANYNAME: First position of any character that can be used in a SAS variable name when VALIDVARNAME is set to V7, returns 0 if absent
ANYPRINT: First position of any printable character, returns 0 if absent
ANYPUNCT: First position of any punctuation character, returns 0 if absent
ANYSPACE: First position of any whitespace character (tabs, carriage returns and the like), returns 0 if absent
ANYUPPER: First position of any uppercase letter, returns 0 if absent
ANYXDIGIT: First position of any hexadecimal character, returns 0 if absent
NOTALNUM: First position of any non-alphanumeric character, returns 0 if absent
NOTALPHA: First position of any non-alphabetic character, returns 0 if absent
NOTCNTRL: First position of anything that isn't a control character, returns 0 if absent
NOTDIGIT: First position of any non-numeric character, returns 0 if absent
NOTFIRST: First position of any character that cannot be used as the start of a SAS variable name when VALIDVARNAME is set to V7, returns 0 if absent
NOTGRAPH: First position of anything that isn't a printable character that isn't white space, returns 0 if absent
NOTLOWER: First position of anything that isn't a lowercase letter, returns 0 if absent
NOTNAME: First position of any character that cannot be used in a SAS variable name when VALIDVARNAME is set to V7, returns 0 if absent
NOTPRINT: First position of any non-printable character, returns 0 if absent
NOTPUNCT: First position of anything that isn't a punctuation character, returns 0 if absent
NOTSPACE: First position of anything that isn't a whitespace character, returns 0 if absent
NOTUPPER: First position of anything that isn't an uppercase letter, returns 0 if absent
NOTXDIGIT: First position of anything that isn't a hexadecimal character, returns 0 if absent
Apart from simpler cases where other techniques would work well with a similar amount of effort, there are others that would need some investigation if you were to program them without using one of the above functions. For that reason, I'll be keeping them in mind for when I might meet one of those more complex scenarios.
Another look at Drupal
20th January 2010Early in the first year of this blog, I got to investigate the use of Drupal for creating an article-based subsite. In the end, the complexities of its HTML and CSS thwarted my attempts to harmonise the appearance of web pages with other parts of the same site, and I discontinued my efforts. In the end, it was Textpattern that suited my needs, and I have stuck with that for the aforementioned subsite.
However, I recently spotted someone very obviously using Drupal in its out of the box state for a sort of blog (there is even an extension for importing WXR files containing content from a WordPress blog); they even hadn't removed the Drupal logo. With my interest rekindled, I took another look for the sake of seeing where things have gone in the last few years. Well, first impressions are that it now looks more like a blogging tool, with greater menu control and the facility to define custom content types.
There are plenty of nice themes around too, though that highlights an idiosyncrasy in the sense that content editing is not fully integrated into the administration area where I'd expect it to be. The consequence of this situation is that pages, posts (or story as the content type is called) or any content types that you have defined yourself are created and edited with the front page theme controlling the appearance of the user interface.
It is made even more striking when you use a different theme for the administration screens. That oddity aside, there is a lot to recommend Drupal, even if I'd try setting up a standalone site with it rather than attempting to shoehorn it as a part of an existing one like what I was trying when I last looked.
Why the manual step?
18th January 2010One of the consequences of buying a new camera is that your current photo processing software may not be fully equipped for the job of handling the images that it creates. This especially manifests itself with raw image files; Adobe Photoshop Elements 5 was unable to completely handle DNG files made with my Pentax K10D until I upgraded to version 7.
As things stood, Elements 7 was unable to import CR2 files from my Canon PowerShot G11 into the Organiser, so it was off to the appropriate page on the Adobe website for a Camera Raw updater. Thus, I picked up the latest release of Camera Raw (5.6 at the time of writing) even though it was found in the Elements 8 category (don't be put by this because release notes address the version compatibility question more extensively).
Strangely, the updater doesn't complete everything because you still need to copy Camera Raw.8bi from the zip archive and backup the original. Quite why this couldn't have been more automated, even with user prompts for file names and locations, is beyond me, yet that is how it is. However, once all was in place, CR2 files were handled by Elements without missing a beat.
A new acquisition
16th January 2010Back in the early days of this blog, I mulled over the idea of having a high-end digital compact camera to complement a DSLR that then was delivering very dusty images; that Canon EOS 10D was cleaned since then and comes in for occasional use to this day. That was nearly three years ago, and a first generation Ricoh GR Digital was the item that then was catching my eye. At the time, I failed to justify spending that much money on such a thing and ended up acquiring a new Pentax K10D DSLR instead. The question that rattled about my head was this: what was the point of spending DSLR money on a compact camera? It is one that never really went away and comes to mind when you see the prices of interchangeable lens compacts like Olympus' Pen and equivalent offerings from Panasonic and Ricoh (there, it's interchangeable lens units rather than actual lenses).
The strongest counterpoint to the cost conundrum is the little matter of size. SLR (film or digital) cameras are sizeable things, and there is a place for having something that drops into a pocket. It is that which has propelled me into taking delivery of a Canon PowerShot G11. It may need a good-sized pocket but, unless you are going out with no jacket, it shouldn't be a problem most of the time. For those shorter sorties when I don't fancy bringing an SLR out, it is well-built and looks the business, though some acclimatisation is to make the best of the knobs, buttons and menus. Nevertheless, the included manual will help with this process (there's a paper quick start guide and more detailed documentation on CD).
The camera hasn't seen extensive use just yet, so here are a few early impressions. Firstly, there's the matter of size: it's even smaller than the first camera that I ever bought (more than fifteen years ago) and that was a Ricoh 35 mm compact film camera. That comparison is even more striking when you consider the feature sets. The Ricoh was a fixed 35 mm lens affair with things like date and time stamping, ISO choice and a nod towards scenic mode selection. In contrast, the much newer Canon is loaded with the sorts of things that normally are found almost exclusively on SLR's, starting with its effective 28-140 mm focal length range.
Exposure modes such as manual, aperture priority and shutter priority complement scene-based modes and another for movies (not a concern of mine, it has to be said). As if that weren't enough, there's exposure compensation too. It came as a surprise to me to find a form of manual focussing included, though it is not as convenient as turning a focussing ring on a lens. While you can see the inbuilt flash above, there's also a hot shoe and a place to attach a tripod too. Settings like white balance and file format are accessed using the Function/Set button, with the lever underneath the shutter release button controlling the focal length of the lens. In addition, there's also image stabilisation, and that's important when you're using live view to compose a photo. Spot metering and focal point selection are other things that find their way into the package. Some may be excited by other things, but exposure and focussing are essential for any photographic efforts.
An optical viewfinder is included, and it has diopter settings too, but my first impressions are that live view through the rear screen trumps it and I see no need for such things on SLR's. That also flips out from the camera body and can be rotated either for self-portraiture or for folding back in on the camera body for use like a non-articulated screen. Another use is with those occasions when the subject means holding the camera in positions that would be impossible with a conventional screen; holding the camera over your head or down low on the ground are the sorts of situations that come to mind.
Of course, there's more there than those features that I have listed, and the specifications on the Canon website are as good a place to start as any. So far, my only testing has taken the form of cursory checks and to make sure that the thing works properly. Still, this has given me more of a feel for the camera and how it operates. As you'd expect, high ISO settings are noisy, only for a bigger surprise to be that the smallest aperture setting is f/8. Being used to SLR's, I was expecting to get the likes of f/16 on there, but a spot of internet investigation indicated that I should have been taking the size of the sensor into account with my expectations. Any trials so far have been in dull weather, so I'd need to use it in a wider variety of conditions before giving it the sort of wider appraisal that you'd find in the likes of Outdoor Photography (who liked it, it has to be said). For what it's worth, I have found no major criticism so far, though I cannot see it usurping my SLR's not as if that ever was the intention anyway.
Understanding Outlook's 32 KB rule limit: What happens when you upgrade from Outlook 2000 to Outlook 2007
12th January 2010A move from Outlook 2000 to Outlook 2007 at work before Christmas resulted in deactivated Outlook rules and messages like the following when I tried reactivating them:
One or more rules could not be uploaded to Exchange server and have been deactivated. This could be because some of the parameters are not supported or there is insufficient space to store all your rules.
The cause is a 32 KB size limitation for all rules associated with your Exchange server account before Exchange 2007. With the latter, the default size increases to 64 KB and can be increased further to 256 KB by manual intervention. Though this wouldn't be a big issue if you had the option to store rules locally on your own PC, but that was removed after Outlook 2000, therefore explaining why I first encountered it when I did.
Microsoft has a useful article on their support website containing suggested remedies which aren't all as extreme as deleting some rules either. Consolidation and shortening of rule names are other suggestions, and you should never discount how much space the "run on this machine only" parameter takes up either. Still, Microsoft made an odd design decision that caused this issue, not that it is their first or even last such choice.
Further securing MySQL in Fedora
4th December 2009Ubuntu users must be spoilt because any MySQL installation asks you for a root password, an excellent thing in my opinion. With Fedora, it just pops the thing on there with you needing to set up a service and setting the root password yourself; if I recall correctly, I think that openSUSE does the same thing. For the service management, I needed to grab system-config-services from the repositories because my Live CD installation left off a lot of stuff, OpenOffice and GIMP even. The following command line recipe addressed the service manager omission:
su - # Change to root, entering password when asked
yum -y install system-config-services # Installs the thing without a yes/no prompt
exit # Return to normal user shell
Thereafter, the Services item from the menus at System > Administration was pressed into service and the MySQL service enabled and started. The next step was to lock down the root user, so the following sequence was used:
mysql # Enter MySQL prompt; no need for user or password because it still is unsecured!
UPDATE mysql.user SET Password=PASSWORD('MyNewPass') WHERE User='root';
FLUSH PRIVILEGES;
quit # Exit the mysql prompt, leaving the bare mysql command unusable
For those occasions when password problems keep you out of the MySQL shell, you'll find password resetting advice on the MySQL website, though I didn't need to go the whole hog here. MySQL Administrator might be another option for this type of thing. That thought never struck me while I was using it to set up less privileged users and allowing them access to the system. For a while, I was well stymied in my attempts to access the MySQL using any of those extra accounts until I got the idea of associating them with a host, another thing that is not needed on Ubuntu if my experience is any guide. All in all, Fedora may make you work a little extra to get things like thing done, yet I am not complaining if it makes you understand a little more about what is going on in the background, something that is never a disadvantage.
Command Line Software Management
2nd December 2009One of the nice things about a Debian-based Linux distribution is that it is easy to pull a piece of software onto your system from a repository using either apt-get or aptitude. While some may prefer a GUI, but I find that the command line offers a certain extra transparency that stops the "what's it doing?" type of question. That's never to say that the GUI-based approach hasn't a place, and I only go using it when seeking out a piece of software without knowing its aptitude-ready name. Interestingly, there are signs that Canonical may be playing with the idea of making Ubuntu's Software Centre a full application management tool with updates and upgrades getting added to the current searching, installation and removal facilities. That well may be, but it's going to take a lot of effort to get me away from the command line altogether.
Fedora and openSUSE have their software management commands too in the shape of yum and zypper, respectively. The recent flurry of new operating system releases has had me experimenting with both of those distros on a real test machine. As might be expected, the usual battery of installation, removal and update activities are well served, and I have been playing with software searching using yum too.
One thing that has yet to mature is in-situ distribution upgrading, à la Ubuntu. In principle, it is possible, but I got a black screen when I tried moving from openSUSE 11.1 to 11.2 within VirtualBox using instructions on the openSUSE website. Not wanting to wait, I reached for a Live CD instead, and that worked a treat on both virtual and real machines.
Being in an experimental turn of mind, I attempted the same to get from Fedora 11 to the beta release of its version 12. A spot of repository trouble got me using a Live CD in its place. You can perform an in-situ upgrade from a full Fedora DVD, but the only option is system replacement when you have a Live CD.
Once installation is out of the way, YAST can be ignored in favour of zypper and yum is good enough that Fedora's GUI-using alternative can be ignored. It's nice to see good transparent ideas taking hold elsewhere and may make migration between distros much easier too.
You always can install things yourself...
26th November 2009With Linux distributions offering you everything on a plate, there is a temptation to stick with what they offer rather than taking things into your own hands. For example, Debian's infrequent stable releases and the fact that they don't seem to change software versions throughout the lifetime of such a release means that things such as browser versions are fixed for the purposes of stability; Lenny has stuck with Firefox 3.06 and called it IceWeasel for some unknown reason. However, I soon got to grab a tarball for 3.5 and popped its contents into /opt where the self-contained package worked without a hitch. The same modus operandi was used to add Eclipse PDT and that applied to Ubuntu too until buttons stopped working, forcing a jumping of ship to NetBeans.
Of course, you could make a mess when veering away from what is in a distribution, but that should be good enough reason not to get carried away with software additions. With the availability of DEB packages for things like Adobe Reader, RealPlayer, VirtualBox, Google Chrome and Opera, keeping things clean isn't so hard. While your mileage may vary when it comes to how well things work out for you, I have only ever had the occasional problem anyway.
What reminded me of this was a recent irritation with the OpenOffice package included in Ubuntu 9.10 whereby spell checking wasn't working. While there were thoughts about in situ fixes like additional dictionary installations, I ended up plumping for what could be called the lazy option: grabbing a tarball full of DEB packages from the OpenOffice website and extracting its contents into /tmp and, once the URE package was in place, installing from there using the command:
dpkg -i o*
To get application shortcuts added to the main menu, it was a matter of diving into the appropriate subfolder and installing from the GNOME desktop extension package. Of course, Ubuntu's OpenOffice variant was removed as part of all this but, if you wanted to live a little more dangerously, the external installation goes into /opt which means that there shouldn't be too much of a conflict anyway. In any case, the DIY route got me the spell checking in OpenOffice Writer that I needed, so all was well and another Ubuntu rough edge eradicated from my life, for now anyway.
Converting from CGM to Postscript
24th November 2009One thing that I recently had to investigate was the possibility of converting CGM vector graphics files into Postscript and from there into PDF. Having used ImageMagick for converting images before, that was an obvious option. However, that cannot process CGM files on its own and needs a delegate or helper application as well. This is the case with raw digital camera files too, with UFRaw being the program chosen. For CGM images, the more obscure RALCGM is what's needed, and tracking it down is a bit of an art. Though the history is that it was developed at the U.K.'s Rutherford Appleton Laboratory, it appears that it was left to go off into the wilderness rather than someone keeping an eye on things. With that in mind, here are the installation packages for Windows and Linux (RPM):
RALCGM is a handy command line tool that can covert from CGM to Postscript on its own without any need for ImageMagick at all. From what I have seen, fonts on graphical output may look greyer than black, but it otherwise does its job well. However, considering that it is a freely available tool, one cannot complain too much. There are other packages for doing vector to raster conversion and the ones that I have seen do have GUI's but the freedom to look at for cost software wasn't mine to have. The required command looks something like the following:
ralcgm -d PS -oL test.cgm test.ps
The switch -d PS uses the software's Postscript driver and -oL specifies landscape orientation. If you like to find out more, here's a PDF rendition of the help file that comes with the thing:
So you just need a web browser?
21st November 2009When Google announced that it was working on an operating system, it was bound to result in a frisson of excitement. However, a peek at the preview edition that has been doing the rounds confirms that Chrome OS is a very different beast from those operating systems to which we are accustomed. The first thing that you notice is that it only starts up the Chrome web browser. In this, it is like a Windows terminal server session that opens just one application. Of course, in Google's case, that one piece of software is the gateway to its usual collection of productivity software like Gmail, Calendar, Docs & Spreadsheets and more. Then, there are offerings from others too, with Microsoft just beginning to come into the fray to join Adobe and many more. As far as I can tell, all files are stored remotely, so I reckon that adding the possibility of local storage and management of those local files would be a useful enhancement.
With Chrome OS, Google's general strategy starts to make sense. First create a raft of web applications, follow them up with a browser and then knock up an operating system. It just goes to show that Google Labs doesn't simply churn out stuff for fun, but that there is a serious point to their endeavours. In fact, you could say that they sucked us in to a point along the way. Speaking for myself, I may not entrust all of my files to storage in the cloud, yet I am perfectly happy to entrust all of my personal email activity to Gmail. It's the widespread availability and platform independence that has done it for me. For others spread between one place and another, the attractions of Google's other web apps cannot be understated. Maybe, that's why they are not the only players in the field either.
With the rise of mobile computing, that kine of portability is the opportunity that Google is trying to use to its advantage. For example, mobile phones are being used for things now that would have been unthinkable a few years back. Then, there's the netbook revolution started by Asus with its Eee PC. All of this is creating an ever internet connected bunch of people, so having devices that connect straight to the web like they would with Chrome OS has to be a smart move. Some may decry the idea that Chrome OS will be available on a device only basis, but I suppose they have to make money from this too; search can only pay for so much, and they have experience with Android too.
There have been some who wondered about Google's activities killing off Linux and giving Windows a good run for its money; Chrome OS seems to be a very different animal to either of these. It looks as if it is a tool for those on the move, an appliance, rather than the pure multipurpose tools that operating systems usually are. If there is a symbol of what an operating system usually means for me, it's the ability to start with a bare desktop and decide what to do next. Transparency is another plus point, with the Linux command line having that in spades. For those who view PC's purely as means to get things done, such interests are peripheral, and it is for these that the likes of Chrome OS has been created. In other words, the Linux community need to keep an eye on what Google is doing but should not take fright because there are other things that Linux always will have as unique selling points. Even though the same sort of thing applies to Windows too, Microsoft's near stranglehold on the enterprise market will take a lot of loosening, perhaps keeping Chrome OS in the consumer arena. Counterpoints to that include the use GMail for enterprise email by some companies and the increasing footprint of web-based applications, even bespoke ones, in business computing. In fact, it's the latter that can be blamed for any tardiness in Internet Explorer development. In summary, Chrome OS is a new type of thing rather than a replacement for what's already there. We may find that co-existence is how things turn out, but what it means for Linux in the netbook market is another matter. Only time will tell on that one.