Technology Tales

Installing FreeBSD in a VirtualBox Virtual Machine

With UNIX being the basis of Linux, I have a soft spot for trying out any UNIX that can be installed on a PC. For a while, I had OpenSolaris on the go and even vaguely recall having a look at one of the BSD’s. However, any recent attempt to install one of the latter, and there are quite a few around now, got stymied by some sort of kernel panic caused by using AMD CPU’s. With the return to the Intel fold arising from the upgrade of my main home PC last year, it perhaps was time to try again.

The recent release of FreeBSD 10.0 was the cue and I downloaded a DVD image for a test installation in a VirtualBox virtual machine with 4 GB of memory and a 32 GB virtual hard drive attached (expanding storage was chosen so not all the allocated space has been taken so far). The variant of FreeBSD chosen was the 64-bit x86 one and I set to installing it in there. Though not as pretty in appearance as those in various Linux distros, the installer was not that user unfriendly to me. Mind you, I have experience of installing Arch Linux so that might have acclimatised me somewhat.

Those installation screens ask about the keyboard mapping that you want and I successfully chose one of the UK options. There was limited opportunity for adding extras though there was a short list of few from which I made some selections. User account set up also was on offer and I would have been better off knowing what groups to assign for my personal user account so as to have to avoid needing to log in as root so often following system start up later. Otherwise, all the default options were sufficient.

When the installation process was complete, it was time to boot into the new system and all that was on offer was a command line log in session. After logging in as root, it was time to press pkg into service in order to get a desktop environment in place. The first step was to install X:

pkg install xorg

Then, it was time to install a desktop environment. While using XFCE or KDE were alternatives, I chose GNOME 2 due to familiarity and more extensive instructions on the corresponding FreeBSD handbook page. Issuing the following command added GNOME and all its helper applications:

pkg install gnome2

So that GNOME starts up at the next reboot, some extra steps are needed. The first of these is to add the following line into /etc/fstab:

proc /proc procfs rw 0 0

Then, two lines were needed in /etc/rc.conf:

gdm_enable=”YES”
gnome_enable=”YES”

The first enables the GNOME display manager and the second activates other GNOME programs that are needed for a desktop session to start. With each of these in place, I got a graphical login screen at the next boot time.

With FreeDSB being a VirtualBox Guest, it was time to consult the relevant FreeBSD manual page. Here, there are sections for a number of virtual machine tools so a search was needed to find the one for VirtualBox. VirtualBox support for FreeBSD is incomplete in that there is no installation media for BSD systems though Linux and Solaris are supported along with Windows. Therefore, it is over to the FreeBSD repositories for the required software:

pkg install virtualbox-ose-additions

Aside from the virtual machine session not capturing and releasing the mouse pointer automatically, that did everything that was needed even if it was the open source edition of the drivers and their proprietary equivalents. To resolve the mouse pointer issue, I needed to temporarily disable the GNOME desktop session in /etc/rc.conf to drop down to a console only session where xorg. conf could be generated using the following commands:

Xorg -configure
cp xorg.conf.new /etc/xorg.conf

In the new xorg.conf file, the mouse section needs to be as follows:

Section “InputDevice”
Identifier  “Mouse0”
Driver      “vboxmouse”
EndSection

If it doesn’t look like the above and it wasn’t the case for me then it needs changing. Also, any extra lines from the default set up also need removing or the mouse will not function as it should. The ALT+F1 (for accessing GNOME menus) and ALT+F2 (for running commands) keyboard shortcuts then become crucial when your mouse is not working as it should and could avert a panic too; knowing that adjusting a single configuration file will fix a problem when doing so is less accessible is not a good feeling as I discovered to my own cost. The graphics settings were fine by default but here’s what you should have in case it isn’t for you:

Section “Device”
### Available Driver options are:-
### Values: <i>: integer, <f>: float, <bool>: “True”/”False”,
### <string>: “String”, <freq>: “<f> Hz/kHz/MHz”
### [arg]: arg optional
Identifier  “Card0”
Driver      “vboxvideo”
VendorName  “InnoTek Systemberatung GmbH”
BoardName   “VirtualBox Graphics Adapter”
BusID       “PCI:0:2:0”
EndSection

The next step is to ensure that your HAL settings are as they should. I needed to create a file in /usr/local/etc/hal/fdi/policy called 90-vboxguest.fdi that contains the following:

<?xml version=”1.0″ encoding=”utf-8″?>
<!--
# Sun VirtualBox
# Hal driver description for the vboxmouse driver
# $Id: chapter.xml,v 1.33 2012-03-17 04:53:52 eadler Exp $
Copyright (C) 2008-2009 Sun Microsystems, Inc.
This file is part of VirtualBox Open Source Edition (OSE, as
available from http://www.virtualbox.org. This file is free software;
you can redistribute it and/or modify it under the terms of the GNU
General Public License (GPL) as published by the Free Software
Foundation, in version 2 as it comes in the “COPYING” file of the
VirtualBox OSE distribution. VirtualBox OSE is distributed in the
hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa
Clara, CA 95054 USA or visit http://www.sun.com if you need
additional information or have any questions.
-->
<deviceinfo version=”0.2″>
<device>
<match key=”info.subsystem” string=”pci”>
<match key=”info.product” string=”VirtualBox guest Service”>
<append key=”info.capabilities” type=”strlist”>input</append>
<append key=”info.capabilities” type=”strlist”>input.mouse</append>
<merge key=”input.x11_driver” type=”string”>vboxmouse</merge>
<merge key=”input.device” type=”string”>/dev/vboxguest</merge>
</match>
</match>
</device>
</deviceinfo>

With all that set, it is time to ensure that the custom user account is added to the wheel and operator groups using this command:

pw user mod [user name] -G wheel operator

Executing the above as root means that the custom account can run the su command so that logging in as root at the start of a desktop session no longer is needed. That is what being in the wheel group allows and the anyone in the operator group can shut down or restart the system. Both are facilities readily available in Linux so I fancied having them in FreeBSD too.

Being able to switch to root in a terminal session meant that I could go on to add software like Firefox, Libreoffice, GIMP, EMACS, Geany, Netbeans, Banshee and so on. There may be a line of opinion that FreeBSD is a server operating system but all of these make it more than passable for serving as a desktop one too. There may be no package management GUI as such and the ones that come with GNOME do not work either but anyone familiar with command line working will get around that.

FreeBSD may be conservative but that has its place too and being able to build up a system one item at a time teaches far more than getting everything already sorted in one hit. So far, there is enough documentation to get me going and I hope to see where else things go too. So far, the OS hasn’t been that intimidating and that’s good to see.

A second post today?

I know what I said about a post every two days but something has entered my head that seems timely. Things seem to starting up for 2008 and my getting a swathe of post ideas is only one of them. Today, Sun has bought up MySQL, the database that stores these ponderings for posterity, and Oracle has finally got its hands on Bea, the people behind the Weblogic software with which I have had an indirect brush for a lot of 2007.

Getting custom Python imports to work in Visual Studio Code

While I continue to use Spyder as my preferred Python code editor, I also tried out Visual Studio Code. Handily, this Integrated Development Environment also has facilities for working with R and Julia code as well as MarkDown text editing and adding the required extensions is enough for these applications; it helps that there is an unofficial Grammarly extension for content creation.

My Python code development makes use of the Pylance extension and it works a little differently from Spyder when it comes to including files using import statements. Spyder will look into the folder where the base script is located but the default behaviour of Pylance is that it looks in the root path of your workspace. This meant that any code that ran successfully in Spyder failed in Visual Studio Code.

The way around this was to add the required location using the python.analysis.extraPaths setting for the workspace. That meant opening Settings by navigating to File > Preferences > Settings in the menu system and entering python.analysis.extraPaths into the search box. That took me to the section that I needed and I then clicked on Add Item before entering the required path and clicking on the OK button. That was enough to fix the problem and all worked as it should after that.

Using multi-line commenting in Perl to inactivate blocks of code during testing

Recently, I needed to inactivate blocks of code in a Perl script while doing some testing. This is something that I often do in other computing languages so I sought the same in Perl. To do that, I need to use the POD methodology. This meant enclosing the code as follows.

=start

<< Code to be inactivated by inclusion in a comment >>

=cut

The =start line could use any word after the equality sign but it seems that =cut is needed to close the multi-line comment. If this was actual programming documentation, then the comment block should include some meaningful text for use with perldoc but that was not a concern here since the commenting statements would be removed afterwards anyway and it is good practice not to leave commented code in a production script or program to avoid any later confusion.

In my case, this facility allowed me to isolate the code that I needed to alter and test before putting everything back as needed. It also saved time since I did not need to individually comment out every executable line because multiple lines could be inactivated at a time.

A web development toolbox

SAS Data Step Hash Objects and Memory

Using hash objects in SAS data step code offers some great advantages from the speed point of view; having a set of data in memory rather than on disk makes things much faster. However, that means that you need to keep more of an eye on the amount of memory that’s being used. The first thing is to work out how much memory is available and it’s not necessarily the total amount installed on the system or, for that matter, the amount of memory per processor on a multi-processor system. What you really need is the number, in bytes, that is stored in the XMRLMEM system option and here’s a piece of code that’ll do just that:

data _null_;
mem=getoption('xmrlmem');
put mem;
run;

The XMRLMEM is itself an option that you can only declare in the system call that starts SAS up in the first place and there are advantages to keeping it under control, particularly on large multi-user servers. However, if your hash objects start to exceed what is available, here’s the sort of thing that you can expect to see:

ERROR: Hash object added 49136 items when memory failure occurred.
FATAL: Insufficient memory to execute data step program. Aborted during the EXECUTION phase.
NOTE: The SAS System stopped processing this step because of insufficient memory.
NOTE: SAS set option OBS=0 and will continue to check statements. This may cause NOTE: No observations in data set.

Those messages are a cue for you to learn to keep those hash objects and to only ever make them as large as your memory settings will allow. Another thing to note is that hash objects are best retained for rather fixed data volumes instead of ones that could outgrow their limits. There’s a certain amount of common sense in operation here but it may be that promoters of hash objects don’t mention their limitations as much as they should. If you want to find out more, SAS have a useful paper on their website and the their Knowledge Base has more on the error messages that you can get.

Selecting SAS code in the Program Editor on UNIX

Here’s a possible bugbear with programming using the SAS Display Manager in UNIX, selecting sections of code and running them. In the installations that I have encountered, the mouse selection is not retained so the code selection cannot be run. There is a fix for this and it is not the most obvious. Going to the Preferences dialogue box (Tools > Options > Preferences… from the menu bar) and selecting the Editing tab brings up the screen below:

Ensuring that "Automatically store selection" is switched off, as shown above, will allow one to select and submit sections of code from a SAS program like what is normal practice with Windows SAS. It isn’t an obvious solution but it does the trick for me.

Another way to look through WordPress code

At times in the past, I have complained about there not being sufficient documentation of the actual code used in WordPress itself. True, there is the venerable Codex but that can be incomplete and any extra help is always useful. Peter Westwood (a.k.a westi) has generated documentation from the code itself and put it up on the web for all to peruse. While we are on the subject of information on the more advanced aspects of WordPress, blogs by Mark Jaquith and Andrew Ozz are also worth a look. As the cliché goes, every little helps…

Sorting out a system update failure for FreeBSD

With my tendency to apply Linux updates using the command, I was happy to see that something similar was possible in FreeBSD too. The first step is to fire up a terminal session and drop into root using the su command. That needs the root superuser password in order to continue and the next step is to update the local repositories using the following command:

pkg update

After that, it is time download updated packages and install these by issuing this command:

pkg upgrade

Most of the time, that is sufficient but I discovered that there are times when the above fails and additional interventions are needed. What I had uncovered were dependency error messages and I set to looking around the web for remedies to this. One forum question that was similar to what I had met with the suggestion of consulting the file called UPDATING in /usr/ports/. An answer like that looks unhelpful but for the inclusion of advice where extra actions were needed. Also, there is a useful article on updating FreeBSD ports that gives more in the way of background knowledge so you understand the more about what needs doing.

Following both that and the UPDATING  file resulted in my taking the following sequence of steps. The first act was to download and initialise the Ports Collection, a set of build instructions.

portsnap fetch extract

The above is a one time only action so future updates are done as follows:

portsnap fetch update

With an up to date Ports Collection in place, it was time to install portman:

pkg install portman

A look through /usr/ports/UPDTAING revealed the commands I needed for updating Python and Perl to address the dependency problem that I was having:

portmaster -o devel/py-setuptools27 devel/py-setuptools
portmaster -r py\*setuptools

With those completed, I re-ran pkg update again and all was well. The extra actions needed to get that result will not get forgotten and I am sharing them on here so I know where they are. If anyone else has use for them, that would be even better.

Avoiding errors caused by missing Julia packages when running code on different computers

As part of an ongoing move to multi-location working, I am sharing scripts and other artefacts via GitHub. This includes Julia programs that I have. That has led me to realise that a bit of added automation would help iron out any package dependencies that arise. Setting up things as projects could help, yet that feels a little too much effort for what I have. Thus, I have gone for adding extra code to check on and install any missing packages instead of having failures.

For adding those extra packages, I instate the Pkg package as follows:

import Pkg

While it is a bit hackish, I then declare a single array that lists the packages to be checked:

pkglits =["HTTP", "JSON3", "DataFrames", "Dates", "XLSX"]

After that, there is a function that uses a try catch construct to find whether a package exists or not, using the inbuilt eval macro to try a using declaration:

tryusing(pkgsym) = try
@eval using $pkgsym
return true
catch e
return false
end

The above function is called in a loop that both tests the existence of a package and, if missing, installs it:

for i in 1:length(pkglits)
rslt = tryusing(Symbol(pkglits[i]))
if rslt == false
Pkg.add(pkglits[i])
end
end

Once that has completed, using the following line to instate the packages required by later processing becomes error free, which is what I sought:

using HTTP, JSON3, DataFrames, Dates, XLSX