12th March 2024
As part of my backup workflow, I maintain a machine running OpenMediaVault that I only power up when backups are to be performed. Typically, this often happens when I have new photography images to load, and I have a NAS that acts as an online backup system. The OpenMediaVault machine is a near-offline counterpart to the NAS for added safety.
Recently, I needed to check on the number of image files in a directory from an SSH session because of a need to create a new repository for 2024. Some files from this year had ended up in the 2023 one, and I needed to be sure that nothing from last year ended in the 2024 folder, or vice versa. Getting a file count from a trusted source was a quick way of doing exactly this.
Due to clumsiness with the NAS, I had to do this using the OpenMediaVault machine. While I could go mounting drives on an interim basis, it was quicker to work from a BASH session. The trick was to use the wc
command for counting the lines output by an invocation of the ls
command. An example follows:
ls -l | wc -l
The -l (as in l
for Lima) switch forces wc
to count lines, while the counterpart (same letter) for ls forces it to list the contents in long form, one item per line. Thus, counting the number of lines gets you the count of the number of files. The call to the ls
command can be customised to add other things life the number of dot files, but the above was enough for my purposes. When the files in both 2023 directories matched, I was satisfied that all was in order.
18th June 2007
The last week on WordPress.com has featured decisions that some may question, to say the least. For instance, the death of Feed Stats was something that I saw as a retrograde step. Next, I saw a slight change to the appearance of my blog that led me to take full control of the situation; a previous discussion with WordPress.com staff about changing a theme to the way that I wanted it to be got me nowhere so I wasn’t about to try again… I might miss having advance notice of where WordPress goes next but I am not sure that I want to be a guinea pig either.
So, you now find this hosted by Streamline.net and, apart from a lengthy hiatus (at least, it felt like forever thanks to recurring thoughts of PageRank loss; yes, I know that sounds silly…) spent awaiting FTP access to be sorted out, the set up ran smoothly enough; I think that my request for transfer from a Windows server to a Linux one might have been the cause of the delay. Setting up a MySQL database was a breeze and it’s part of the package too. In fact, the Pro package that I am using is £31.99 ex VAT per annum, not bad at all, and, teething problems out of the way, it’ll be interesting to see how things will fare from now on.
Having a self-hosted WordPress installation is nothing new to me since I do it for my hillwalking blog and everything came together very quickly this time around. knowing what plug-ins add real value was a definite help: Bad Behaviour and Ultimate GA headed the list but a Spam Karma 2 is another option. Feedburner integration is another potential item on the configuration list. I have stuck with the Andreas09 theme but am unsure as to how far I will take customising it; it’s not a high priority right now. In fact, I may find another three-column layout that takes my eye now that I am not limited to the offerings available from WordPress.com.
The posts from the old blog have arrived over here and that seems to have worked fine first time around thanks the WordPress’ import/export functionality. I still have to get the images over but there’ll be time for that yet. Another thing on the to do list is to transfer over the links and set up any text and RSS widgets that have been a feature of the previous rendition of the blog. That means linking to wp-links-opml.php on the old blog using the import links functionality. There is no other link export function and you can only import into a single category; a link import/export plug-in that retains the link category information would be a bonus.
While I wonder if I have made a rod for my own back with my having two self-hosted WordPress blogs, it does feel good to have more control and it’ll be interesting where where this journey goes next.
18th February 2022
While I continue to use Spyder as my preferred Python code editor, I also tried out Visual Studio Code. Handily, this Integrated Development Environment also has facilities for working with R and Julia code as well as MarkDown text editing and adding the required extensions is enough for these applications; it helps that there is an unofficial Grammarly extension for content creation.
My Python code development makes use of the Pylance extension and it works a little differently from Spyder when it comes to including files using import statements. Spyder will look into the folder where the base script is located but the default behaviour of Pylance is that it looks in the root path of your workspace. This meant that any code that ran successfully in Spyder failed in Visual Studio Code.
The way around this was to add the required location using the python.analysis.extraPaths setting for the workspace. That meant opening Settings by navigating to File > Preferences > Settings in the menu system and entering python.analysis.extraPaths into the search box. That took me to the section that I needed and I then clicked on Add Item before entering the required path and clicking on the OK button. That was enough to fix the problem and all worked as it should after that.
26th December 2019
Recently, I needed to inactivate blocks of code in a Perl script while doing some testing. This is something that I often do in other computing languages so I sought the same in Perl. To do that, I need to use the POD methodology. This meant enclosing the code as follows.
=start
<< Code to be inactivated by inclusion in a comment >>
=cut
The =start line could use any word after the equality sign but it seems that =cut is needed to close the multi-line comment. If this was actual programming documentation, then the comment block should include some meaningful text for use with perldoc but that was not a concern here since the commenting statements would be removed afterwards anyway and it is good practice not to leave commented code in a production script or program to avoid any later confusion.
In my case, this facility allowed me to isolate the code that I needed to alter and test before putting everything back as needed. It also saved time since I did not need to individually comment out every executable line because multiple lines could be inactivated at a time.
3rd June 2008
Using hash objects in SAS data step code offers some great advantages from the speed point of view; having a set of data in memory rather than on disk makes things much faster. However, that means that you need to keep more of an eye on the amount of memory that’s being used. The first thing is to work out how much memory is available and it’s not necessarily the total amount installed on the system or, for that matter, the amount of memory per processor on a multi-processor system. What you really need is the number, in bytes, that is stored in the XMRLMEM system option and here’s a piece of code that’ll do just that:
data _null_;
mem=getoption('xmrlmem');
put mem;
run;
The XMRLMEM is itself an option that you can only declare in the system call that starts SAS up in the first place and there are advantages to keeping it under control, particularly on large multi-user servers. However, if your hash objects start to exceed what is available, here’s the sort of thing that you can expect to see:
ERROR: Hash object added 49136 items when memory failure occurred.
FATAL: Insufficient memory to execute data step program. Aborted during the EXECUTION phase.
NOTE: The SAS System stopped processing this step because of insufficient memory.
NOTE: SAS set option OBS=0 and will continue to check statements. This may cause NOTE: No observations in data set.
Those messages are a cue for you to learn to keep those hash objects and to only ever make them as large as your memory settings will allow. Another thing to note is that hash objects are best retained for rather fixed data volumes instead of ones that could outgrow their limits. There’s a certain amount of common sense in operation here but it may be that promoters of hash objects don’t mention their limitations as much as they should. If you want to find out more, SAS have a useful paper on their website and the their Knowledge Base has more on the error messages that you can get.
5th June 2007
Here’s a possible bugbear with programming using the SAS Display Manager in UNIX, selecting sections of code and running them. In the installations that I have encountered, the mouse selection is not retained so the code selection cannot be run. There is a fix for this and it is not the most obvious. Going to the Preferences dialogue box (Tools > Options > Preferences… from the menu bar) and selecting the Editing tab brings up the screen below:

Ensuring that "Automatically store selection" is switched off, as shown above, will allow one to select and submit sections of code from a SAS program like what is normal practice with Windows SAS. It isn’t an obvious solution but it does the trick for me.
10th August 2008
At times in the past, I have complained about there not being sufficient documentation of the actual code used in WordPress itself. True, there is the venerable Codex but that can be incomplete and any extra help is always useful. Peter Westwood (a.k.a westi) has generated documentation from the code itself and put it up on the web for all to peruse. While we are on the subject of information on the more advanced aspects of WordPress, blogs by Mark Jaquith and Andrew Ozz are also worth a look. As the cliché goes, every little helps…
4th December 2013
Recently, I needed to put in place some code to detect the presence or absence of a variable in a dataset and I chose SAS Macro programming as the way to do what I wanted. The logic was based on a SAS sample that achieved the same result in a data step and some code that I had for detecting the presence or absence of a dataset. Mixing the two together gave me something like the following:
%macro testvar(ds=,var=);
%let dsid=%sysfunc(open(&ds,in));
%let varexist=%sysfunc(varnum(&dsid,&var));
%if &dsid > 0 %then %let rc=%sysfunc(close(&dsid));
%if &varexist gt 0 %then %put Info: Variable &var is in the &ds dataset;
%else %put Info: Variable &var is not in the &ds dataset;
%mend testvar;
%testvar(ds=dataset,var=var);
What this does is open up a dataset and look for the variable number in the dataset. In datasets, variables are numbered from left to right with 1 for the first one, 2 for the second and so on. If the variable is not in the dataset, the result is 0 so you know that it is not there. All of this is what the VARNUM SCL function within the SYSFUNC macro function does. In the example, this resolves to %sysfunc(varnum(&dsid,var)) with no quotes around the variable name like you would do in data step programming. Once you have the variable number or 0, then you can put in place some conditional logic that makes use of the information like what you see in the above simple example. Of course, that would be expanded to something more useful in real life but I hope it helps to show you the possibilities here.
17th December 2024
When using VSCode on Linux Mint, I noticed that it was not finding any Python package installed into my user area, as is my usual way of working. Thus, it was being highlighted as being missing when it was already there.
The solution was to navigate to File > Preferences > Settings and click on the Open Settings (JSON) icon in the top right-hand corner of the app to add something like the following snippet in there.
"python.analysis.extraPaths": [
"/home/[user]/.local/bin"
]
Once you have added your user account to the above, saving the file is the next step. Then, VSCode needs a restart to pick up the new location. After that, the false positives get eliminated.
12th February 2007
Over the weekend, I have been updating the theme on my other blog, HennessyBlog. It has been a task that projected me onto a learning curve with the WordPress 2.1 codebase. I have collected what I encountered so I know that it’s out there on the web for you (and I) to use and peruse. It took some digging to get to know some of what you find below. Any function used to power WordPress takes some finding so I need to find one place on the web where the code for WordPress is fully documented. The sites presenting tutorials on how to use WordPress are more often than not geared towards the non-techie rather than code cutters like myself. Then again, they might be waiting for someone to do it for them…
The changes made are as follows:
Tweaks to the interface
These are subtle with the addition of navigation controls to the sidebar and the change in location of the post metadata being the most obvious enhancements. “Decoration” with solid and dashed lines (using CSS border attributes rather than the deprecated hr tagset) and standards compliance links.
Standards compliance
Adding standards compliance links does mean that you’d better check that all is in order; it was then that I discovered that there was work to be done. There is an issue with the WordPress wpautop function (it lives in the formatting.php file) in that it sometimes doesn’t add closing tags. Finding out that it was this function that is implicated took a trip to the WordPress.org website; a good rummage in the wp-includes folder does a lot but it can’t achieve everything.
Like a lot of things in the WordPress code, the wpautop function isn’t half buried. The the_content function (see template-functions-post.php) used to output blog entries calls get_content function (also in template-functions-post.php) to extract the data from mySQL. The add_filter function (in plugin.php) associates the wpautop function and others with get_the_content function and the p tags get added to the output.
To return to the non-ideal behaviour that caused me to start out on the above quest, an example is where you have an img tag enclosed by div tags. The required substitution involves the use of regular expressions that work most of the time but get confused here. So adding a hack to the wpautop function was needed to change the code so that the p end tag got inserted. I’ll be keeping an eye out for any more scenarios like this that slip through the net and for any side effects. Otherwise, compliance is just making sure that all those img tags have their alt attributes completed.
Tweaks to navigation code
Most of my time has been spent on tweaking of the PHP code supporting the navigation. Different functions were being called in different places and I wanted to harmonise things. To do this, I created new functions in the functions.php for my theme and needed to resolve a number of issues along the way. Not least among these were regular expressions used for subsetting with the preg_match match that weren’t to my eyes Perl-compliant, as would be implied by the choice of function. I have since found that PCRE’s in PHP use a more pragmatic syntax but there remained issues with the expressions that were being used. They seemed to behave OK in their native environment but fell out of favour within the environs of my theme. Being acquainted with Perl, I went for a more familiar expression style and the issue has been resolved.
Along the way, I broke the RSS feed. This was on my off-line test blog so no one, apart from myself, that is, would have noticed. After a bit of searching, I realised that some stray white-space from the end of a PHP file (wp-config.php being a favourite culprit), after the PHP end tag in the script file as it happens, was finding its way into the feed and causing things to fall over. Feed readers don’t take too kindly to the idea of the XML declaration not making an appearance on the first line of the file. The refusal of Firefox to refresh things as it should caused some confusion until I realised that a forced refresh of the feed display was needed -- sometimes, it takes a while for an addled brain to think of these kinds of things.