Adventures & experiences in contemporary technology
Recently, I encountered the following kind of message when reading an Excel file into SAS using PROC IMPORT:
ERROR: Error opening XLSX file -> xxx-.xlsx . It is either not an Excel spreadsheet or it is damaged. Error code=8000101D
Requested Input File Is Invalid
ERROR: Import unsuccessful. See SAS Log for details.
Naturally, thoughts arise regarding the state of the Excel file when you see a message like this but that was not the case because the file opened successfully in Excel and looked OK to me. After searching on the web, I found that it was a file permissions issue. The actual environment that I was using at the time was entimICE and I had forgotten to set up a link that granted read access to the file. Once that was added, the problem got resolved. In other systems, checking on file system permissions is needed even if the message seems to suggest that you are experiencing a file integrity problem.
While some Linux distros like Fedora install VirtualBox drivers during installation time, I prefer to install the VirtualBox Guest Additions themselves. Before doing this, it is best to remove the virtualbox-guest-additions package from Fedora to avoid conflicts. After that, execute the following command to ensure that all prerequisites for the VirtualBox Guest Additions are in place prior to mounting the VirtualBox Guest Additions ISO image and installing from there:
sudo dnf -y install gcc automake make kernel-headers dkms bzip2 libxcrypt-compat kernel-devel perl
During the installation, you may encounter a message like the following:
ValueError: File context for /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf already defined
This is generated by SELinux so the following commands need executing before the VirtualBox Guest Additions installation is repeated:
sudo semanage fcontext -d /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf
sudo restorecon /opt/VBoxGuestAdditions-<VERSION>/other/mount.vboxsf
Without doing the above step and fixing the preceding error message, I had an issue with mounting of Shared Folders whereby the mount point was set up but no folder contents were displayed. This happened even when my user account was added to the vboxsf group and it proved to be the SELinux context issue that was the cause.
For as long as I have been programming with SAS, there has been the ability to test if a variable does or does not have one value from a list of values in data step IF clauses or WHERE clauses in both data step and most if not all procedures. It was only within the last decade that its Macro language got similar functionality with one caveat that I recently uncovered: you cannot have a NOT IN construct. To get that, you need to go about things in a different way.
In the example below, you see the NOT operator being placed before the IN operator component that is enclosed in parentheses. If this is not done, SAS produces the error messages that caused me to look at SAS Usage Note 31322. Once I followed that approach, I was able to do what I wanted without resorting to older more long-winded coding practices.
options minoperator;
%macro inop(x);
%if not (&x in (a b c)) %then %do;
%put Value is not included;
%end;
%else %do;
%put Value is included;
%end;
%mend inop;
%inop(a);
Running the above code should produce a similar result to another featured on here in another post but the logic is reversed. There are times when such an approach is needed. One is where a small number of possibilities is to be excluded from a larger number of possibilities. Programming often involves more inventive thinking and this may be one of those.
What you see above is a an error issued by a SAS program like what a colleague at work recently found. The following code will reproduce this so let us walk through the steps to explain a possible cause for this.
The first stage is to create a test dataset containing variables y and x, for the vertical and midpoint axes, respectively, and populating these using a CARDS statement in a data step:
data a;
input y x;
cards;
1 5
3 9
;
run;
Now, we define an axis with tick marks for particular values that will be used as the definition for the midpoint or horizontal axis of the chart:
axis1 order=(1 3);
Then, we try creating the chart using the GCHART procedure that comes with SAS/GRAPH and this is what results in the error message being issued in the program log:
proc gchart data=a;
vbar x / freq=y maxis=axis1;
run;
quit;
The cause is that the midpoint axis tick marks are no included in the data so changing these to the actual values of the x variable removes the message and allows the creation of the required chart. Thus, the AXIS1 statement needs to become the following:
axis1 order=(5 9);
Another solution is to remove the MAXIS option from the VBAR statement and let GCHART be data driven. However, if requirements do not allow this, create a shell dataset with all expected values for the midpoint axis with y set 0 since that is used for presenting frequencies as per the FREQ option in the VBAR statement.
Once you end up with Windows overwriting your master boot record (MBR), you have lost the ability to use GRUB. Therefore, it would be handy to get it back if you want to start up Linux again. Though the loss of GRUB from the MBR was a deliberate act of mine, I knew that I’d have to restore GRUB to get Linux working again.So, I have been addressing the situation with a Live DVD for the likes of Ubuntu or Linux Mint. Once one of those had loaded its copy of the distribution, issuing the following command in a terminal session gets things back again:
sudo grub-install --root-directory=/media/0d104aff-ec8c-44c8-b811-92b993823444 /dev/sda
When there were error messages, I tried this one to see if I could get more information:
sudo grub-install --root-directory=/media/0d104aff-ec8c-44c8-b811-92b993823444 /dev/sda --recheck
Also, it is possible to mount a partition on the boot drive and use that in the command to restore GRUB. Here is the required combination:
sudo mount /dev/sda1 /mnt
sudo grub-install --root-directory=/mnt /dev/sda
Either of these will get GRUB working without a hitch and they are far more snappy than downloading Boot-Repair and using that; I was doing that for a while until a feature on triple booting appeared in an issue of Linux User & Developer that reminded me of the more readily available option. Once, there was a need to manually add an entry for Windows 7 to the GRUB menu too and, with that instated, I was able to dual-boot Ubuntu and Windows using GRUB to select which one was to start for me. Since then, I have been able to dual boot Linux Mint and Windows 8.1 with GRUB finding the latter all by itself so your experiences too may show this variation so it’s worth bearing in mind.
With my tendency to apply Linux updates using the command, I was happy to see that something similar was possible in FreeBSD too. The first step is to fire up a terminal session and drop into root using the su command. That needs the root superuser password in order to continue and the next step is to update the local repositories using the following command:
pkg update
After that, it is time download updated packages and install these by issuing this command:
pkg upgrade
Most of the time, that is sufficient but I discovered that there are times when the above fails and additional interventions are needed. What I had uncovered were dependency error messages and I set to looking around the web for remedies to this. One forum question that was similar to what I had met with the suggestion of consulting the file called UPDATING in /usr/ports/. An answer like that looks unhelpful but for the inclusion of advice where extra actions were needed. Also, there is a useful article on updating FreeBSD ports that gives more in the way of background knowledge so you understand the more about what needs doing.
Following both that and the UPDATING file resulted in my taking the following sequence of steps. The first act was to download and initialise the Ports Collection, a set of build instructions.
portsnap fetch extract
The above is a one time only action so future updates are done as follows:
portsnap fetch update
With an up to date Ports Collection in place, it was time to install portman:
pkg install portman
A look through /usr/ports/UPDTAING revealed the commands I needed for updating Python and Perl to address the dependency problem that I was having:
portmaster -o devel/py-setuptools27 devel/py-setuptools
portmaster -r py\*setuptools
With those completed, I re-ran pkg update again and all was well. The extra actions needed to get that result will not get forgotten and I am sharing them on here so I know where they are. If anyone else has use for them, that would be even better.
Installing the latest version of Citrix Receiver (13.0 at the time of writing) on 64-bit Ubuntu should be as simple as downloading the required DEB package and double-clicking on the file so that Ubuntu Software Centre can work its magic. Unfortunately, the 64-bit DEB file is faulty so that means that the Ubuntu community how-to guide for Citrix still is needed. In fact, any user of Linux Mint or another distro that uses Ubuntu as its base would do well to have a look at that Ubuntu link.
For sake of completeness, I still am going to let you in on the process that worked for me. Once the DEB file has been downloaded, the first task is to creating a temporary folder where the DEB file’s contents can be extracted:
mkdir ica_temp
With that in place, it then is time to do the extraction and it needs two commands with the second of these need to extract the control file while the first extracts everything else.
sudo dpkg-deb -x icaclient- ica_temp
sudo dpkg-deb --control icaclient- ica_temp/DEBIAN
It is the control file that has been the cause of all the bother because it refers to unavailable dependencies that it really doesn’t need anyway. To open the file for editing, issue the following command:
sudo gedit ica_temp/DEBIAN/control
Then change line 7 (it should begin with Depends:) to: Depends: libc6-i386 (>= 2.7-1), lib32z1, nspluginwrapper. There are other software packages in there that Ubuntu no longer supports and they are not needed anyway. With the edit made and the file saved, the next step is to build a new DEB package with the corrected control file:
dpkg -b ica_temp icaclient-modified.deb
Once you have the package, the next step is to install it using the following command:
sudo dpkg -i icaclient-modified.deb
If it fails, then you have missing dependencies and the following command should sort these before a re-run of the above command again:
sudo apt-get install libmotif4:i386 nspluginwrapper lib32z1 libc6-i386
With Citrix Receiver installed, there is one more thing that is needed before you can use it freely. This is to put Thawte security certificate files into /opt/Citrix/ICAClient/keystore/cacerts. What I had not realised until recently was that many of these already are in /usr/share/ca-certificates/mozilla and linking to them with the following command makes them available to Citrix receiver:
sudo ln -s /usr/share/ca-certificates/mozilla/* /opt/Citrix/ICAClient/keystore/cacerts/
Another approach is to download the Thawte certificates and extract the archive to /tmp/. From there they can be copied to /opt/Citrix/ICAClient/keystore/cacerts and I copied the Thawte Personal Premium certificate as follows:
sudo cp /tmp/Thawte Root Certificates/Thawte Personal Premium CA/Thawte Personal Premium CA.cer /opt/Citrix/ICAClient/keystore/cacerts/
Until I found out about what was in the Mozilla folder, I simply picked out the certificate mentioned in the Citrix error message and copied it over like the above. Of course, all of this may seem like a lot of work to those who are non-tinkerers and I have added a repaired 64-bit DEB package that incorporates all of the above and should not need any further intervention aside from installing it using GDebi, Ubuntu’s Software Centre, dpkg or anything else that does what’s needed.
With its final release being near at hand, I decided to have a look at the beta release of Ubuntu GNOME 13.10 to get a sense of what might be coming. A misstep along the way had me inadvertently download and install the 64-bit edition of 13.04 into a VirtualBox virtual machine. The intention to update that to its soon to be released successor was scuppered by instability so I never did get to try out an in situ upgrade to 13.10. What I had in mind was to issue the following command:
gksu update-manager -d
However, I found another one when considering how Ubuntu Server might be upgraded without the GUI application that is the Update Manager. To update to a development version, the following command is what you need:
sudo do-release-upgrade -d
To upgrade to a final release of of a new version of Ubuntu, drop the -d switch from the above to use the following:
sudo do-release-upgrade
There is one further option that isn’t recommended for moving between Ubuntu versions but I use it to get updates such as new kernel subversions that are released:
sudo apt-get dist-upgrade
Rather than trying out the above, I downloaded the latest ISO image for the beta release of Ubuntu GNOME 13.10 and installed onto a VM that instead. Though it is the 32 bit version of the distro that is installed on my main home PC, it has been the 64 bit version that I have been trying. So far, that seems to be behaving itself even if it feels a little sluggish but that could be down to the four year old PC that hosts the virtual machine. For a while, I have been playing with the possibility of an upgrade involving an Intel Core i5 4670K CPU and 16 GB of RAM (useful for running multiple virtual machines at a time) along with any motherboard that supports those so looking at a 64 bit operating system has its uses.
The Linux kernel may be 3.11 but that is not my biggest concern. Neither is the fact that LibreOffice 4.1.2.3 was included and GIMP wasn’t, especially when that could be added easily anyway and it is version 2.8.6 that you get. The move to GNOME Shell 3.8 was what drew me to seeing what was coming because I have been depending on a number extensions. As with WordPress and plugins, GNOME Shell seems to have a tempestuous relationship with some of its extensions and I wanted to see which ones still worked. There also has been a change to the backstage application view in that you either get all installed applications displayed when you browse them or you have to start typing the name of the one you want to select it. Losing the categorical view that has been there until GNOME Shell 3.6 is a step backwards and I hope that version 3.10 has seen some sort of a reinstatement. There is a way to add these categories and the result is not as it once was either; also, it shouldn’t be necessary for anyone to dive into a systems innards to address things like this. With all the constant change, it is little wonder that Cinnamon has become a standalone entity with the release of its version 2.0 and that Debian’s toyed with not going with GNOME for its latest version (7.1 at the time of writing and it picked a good GNOME Shell version in 3.4).
Having had a look at other distribution that already have GNOME Shell 3.8, I knew that a few of my extensions worked with it. The list includes Frippery Bottom Panel, Frippery Move Clock, Places Status Indicator, Removable Drive Menu, Remove Rounded Corners (not really needed with the GNOME Shell theme that I use, Elementary Luna 3.4, but I retain it anyway), Show Desktop Button, User Themes and Ignore_Request_Hide_Titlebar. Because of the changes to the backstage view, I added Frippery Applications Menu in preference to Applications Menu because I have found that to be unstable. Useful new discoveries have included Curtains Up and GNOME Shell Open Terminal while Shell Restart User Menu Entry has made a return and found a use this time around too.
There have been some extensions that were not updated to work with GNOME Shell 3.8 that I have got working. In some cases, it was as simple as updating the metadata.json file for an extension with new version numbers of 3.8 and 3.84 to the list associated with the shell version property. All extensions are to be found in the .local/share/gnome-shell/extensions location in your home directory and each has a dedicated file containing the aforementioned file.
With others, it was a matter of looking in the Looking Glass (execute lg in the box that ALT + F2 brings up on your screen to access this) and seeing what error messages were to be found in there before attempting to correct these in either the extensions’ extension.js files or whatever JavaScript (*.js) file was causing the problem. With either or both of these remedies, I managed to port the four extensions below to GNOME Shell 3.8. In fact, you can download these zip files and install them yourself to see how you get on with them.
Advanced Settings in User Menu
There is a Remove Panel App Menu that works with GNOME Shell 3.8 but I found that it got rid of the Places menu instead of the panel’s App Menu so I tried porting the older extension to see if it behaved itself and it does. With these in place, I have bent Ubuntu GNOME 13.10 to my will ahead of its final release next week and that includes customising Nautilus too. Other than a new version of GNOME Shell, it looks as if it will come with less in the way of drama and a breather like that is no bad thing given that personal computing continues to remain in a state of flux these days.
One of my PC’s has ended up becoming a testing ground for a number of Linux distributions. The list has included openSUSE, Fedora, Arch and LMDE with Sabayon being the latest incumbent. From Arch onwards in that list though, a message has appeared on loading the desktop with every one of these when I have Dropbox’s client set up on there:
Unable to monitor entire Dropbox folder hierarchy. Please run “echo 100000 | sudo tee /proc/sys/fs/inotify/max_user_watches” and restart Dropbox to correct the problem.
Even applying the remedy that the message suggests won’t permanently fix the problem. For that, you need to edit /etc/sysctl.conf with superuser access and add the following line to it:
fs.inotify.max_user_watches = 100000
With that in place, you can issue the following command to fix the problem in the current session (assuming your user account is listed in /etc/sudoers):
sudo sysctl -p & dropbox stop & dropbox start
A reboot should demonstrate that the messages no longer appear again. For a good while, I had ignored it but curiosity eventually got me to find out how it could be stopped and led to what you find above.
A habit of mine is to put a LENGTH or ATTRIB statement between DATA and SET statements in a SAS data step to reset variable lengths. By default, it seems that this triggers truncation warnings in SAS 9.2 or SAS 9.3 when it didn’t in previous versions. SAS 9.1.3, for instance, allowed you have something like the following for shortening a variable length without issuing any messages at all:
data b;
length x $100;
set a;
run;
In this case, x could have a length of 200 previously and SAS 9.1.3 wouldn’t have complained. Now, SAS 9.2 and 9.3 will issue a warning if the new length is less than the old length. This can be useful to know but it can be changed using the VARLENCHK system option. The default value is WARN but it can be set to ERROR if you really want to ensure that there is no chance of truncation. Then, you get error messages and the program fails where it normally would run with warnings. Setting the value of the option to NOWARN restores the type of behaviour seen in SAS 9.1.3 and versions prior to that.
The SAS documentation says that the ability to change VARLENCHK can be restricted by an administrator so you might need to deal with this situation in a more locked down environment. Then, one option would be to do something like the following:
data b;
drop x;
rename _x=x;
set a;
length _x $100;
_x=strip(x);
run;
It’s a bit more laborious than setting to the VARLENCHK option to NOWARN but the idea is that you create a new variable of the right length and replace the old one with it. That gets rid of warnings or errors in the log and resets the variable length as needed. Of course, you have to ensure that there is no value truncation with either remedy. If any is found, then the dataset specification probably needs updating to accommodate the length of the values in the data. After all, there is no substitute for getting to know your data and doing your own checking should you decide to take matters into your hands.
There is a use for the default behaviour though. If you use a specification to specify a shell for a dataset, then you will be warned when the shell shortens variable lengths. That allows you to either adjust the dataset or your program. Also, it gives more information when you get variable length mismatch warnings when concatenating or merging datasets. There was a time when SAS wasn’t so communicative in these situations and some investigation was needed to establish which variable was affected. Now, that has changed without leaving the option to work differently if you so do desire. Sometimes, what can seem like an added restriction can have its uses.