Technology Tales

Adventures & experiences in contemporary technology

Creating soft and hard symbolic links using the Windows command line

19th August 2015

In the world of UNIX and Linux, symbolic links are shortcuts but they do not work like normal Windows shortcuts because you do not jump from one location to another with the file manager’s address bar changing what it shows. Instead, it is as if you see the contents of the directory at another quicker to access location in the file system and the same sort of thinking applies to files too. In some ways, it is like giving files and directories alternative aliases. There are soft links that point to the name of a given directory or file and hard links that point to actual files or directories.

For a long time, I was under the mistaken impression that such things did not exist in Windows until I came across the mklink command, which came with the launch of Windows Vista at the start of 2007. While the then new feature may not be an obvious to most of us, it does does that Windows did incorporate a feature from UNIX and Linux well before the advent of virtual desktops on Windows 10.

By default, the aforementioned command sets up symbolic links to files and the /D switch allows the same to be done for directories too. The /H switch makes a hard link instead of a soft link so we get much of the functionality of the ln command in UNIX and Linux. Here is an example that creates a soft symbolic link for a directory:

mklink /D shortcut target_directory

Above, shortcut is the name of the symbolic link file and target_directory is the destination to which it links. In my experience, it works best for destinations beyond your home folder and, from what I have read, hard links may not be possible across different disks either.

A few more shell commands

8th July 2015

Here are some Linux commands that I encountered in a feature article in the current issue of Linux User & Developer that I had not met before:

cd --

This returns you to the previous directory where you were before with having to go back through the folder hierarchy to get there and is handy if you are jumping around a file system and any other means is far from speedy.

lsb_release -a

It can be useful to uncover what version of a distro you have from the command line and the above works for distros as diverse as Linux Mint, Debian, Fedora (it automatically installs in Fedora 22 if it is not installed already, a more advanced approach than showing you the command like in Linux Mint or Ubuntu), openSUSE and Manjaro. These days, the version may not change too often but it still is good to uncover what you have.

yum install fedora-upgrade

This one can be run either with sudo or in a root session started with su and it is specific to Fedora. The command performs an upgrade of the Fedora distro itself and I wonder if the functionality has been ported to the dnf command that has taken over from yum. My experiences with that in Fedora 22 so far suggest that it should be the case though I need to check that further with the VirtualBox VM that I have created.

Smarter file renaming using PowerShell

14th November 2014

It seems that the Rename-Item commandlet in Powershell is a very useful tool when it comes to smarter renaming of files. Even text substitution is a possibility and what follows is an example that takes the output of the Dir command for listing the files in a directory and replaces hyphens with underscores in each one.

Dir | Rename-Item –NewName { $_.name –replace “-“,”_” }

The result is that something like the-file.txt becomes the_file.txt. This behaviour is reminiscent of the rename command found on Linux and UNIX systems, where regular expressions can be used like in the following example that has the same result as the above:

rename ‘s/-/_/g’ *

In both cases, you do need to be careful as to what files are in a directory for this though the wildcard syntax on Linux or UNIX will be more familiar who has worked with files via almost any command line. Another thing to watch in the UNIX world is that ** parses the whole directory structure and that could be something that is not wanted for much of the time.

All of this is a far cry from the capabilities of the ren or rename command used in the days of MS-DOS and what has become the legacy Windows command line. Apart from simple renaming, any attempt at tweaking a filename through substitution ended up with the extra string getting appended to filenames when I tried it. Thus, the Powershell option looks better in comparison.

Changing file timestamps using Windows PowerShell

29th October 2014

Recently, a timestamp got changed on an otherwise unaltered file on me and I needed to change it back. Luckily, I found an answer on the web that used PowerShell to do what I needed and I am recording it here for future reference. The possible commands are below:

$(Get-Item temp.txt).creationtime=$(Get-Date "27/10/2014 04:20 pm")
$(Get-Item temp.txt).lastwritetime=$(Get-Date "27/10/2014 04:20 pm")
$(Get-Item temp.txt).lastaccesstime=$(Get-Date "27/10/2014 04:20 pm")

The first of these did not interest me since I wanted to leave the file creation date as it was. The last write and access times were another matter because these needed altering. The Get-Item commandlet brings up the file, so its properties can be set. Here, these include creationtime, lastwritetime and lastaccesstime. The Get-Date commandlet reads in the provided date and time for use in the timestamp assignment. While PowerShell itself is case-insensitive, I have opted to show the camel case that is produced when you are tabbing through command options for the sake of clarity.

The Get-Item and Get-Date have aliases of gi and gd, respectively and the Get-Alias commandlet will show you a full list while Get-Command (gcm) gives you a list of commandlets. Issuing the following gets you a formatted list that is sent to a text file:

gcm | Format-List > temp2.txt

There is some online help but it is not quite as helpful as it ought to be so I have popped over to Microsoft Learn whenever I needed extra enlightenment. Here is a command that pops the full thing into a text file:

Get-Help Format-List -full > temp3.txt

In fact, getting a book might be the best way to find your way around PowerShell because of all its commandlets and available objects.

For now, other commands that I have found useful include the following:

Get-Service | Format-List
New-Item -Name test.txt -ItemType "file"

The first of these gets you a list of services while the second creates a new blank text file for you and it can create new folders for you too. Other useful commandlets are below:

Get-Location (gl)
Set-Location (sl)
Copy-Item
Remove-Item
Move-Item
Rename-Item

The first of the above is like the cwd or pwd commands that you may have seen elsewhere in that the current directory location is given. Then, the second will change your directory location for you. After that, there are commandlets for copying, deleting, moving and renaming files. These also have aliases so users of the legacy Windows command line or a UNIX or Linux shell can use something that is familiar to them.

Little fixes like the one with which I started this piece are all very good to know but it is in scripting that PowerShell really is said to show its uses. Having seen the usefulness of such things in the world on Linux and UNIX, I cannot disagree with that and PowerShell has its own IDE too. That may be just as well given how much there is to learn. That especially is the case when you might need to issue the following command in a PowerShell session opened using the Run as Administrator option just to get the execution as you need it:

Set-ExecutionPolicy RemoteSigned

Issuing Get-ExecutionPolicy will show you if this is needed when the response is: Restricted. A response of RemoteSigned shows you that all is in order, though you need to check that any script you then run has no nasty payload in there, which is why execution is restrictive in the first place. This sort of thing is yet another lesson to be learnt with PowerShell.

Changing the working directory in a SAS session

12th August 2014

It appears that PROC SGPLOT and other statistical graphics procedures create image files even if you are creating RTF or PDF files. By default, these are PNG files but there are other possibilities. When working with PC SAS , I have seen them written to the current working directory and that could clutter up your folder structure, especially if they are unwanted.

Being unable to track down a setting that controls this behaviour, I resolved to find a way around it by sending the files to the SAS work directory so they are removed when a SAS session is ended. One option is to set the session’s working directory to be the SAS work one and that can be done in SAS code without needing to use the user interface. As a result, you get some automation.

The method is implicit though in that you need to use an X statement to tell the operating system to change folder for you. Here is the line of code that I have used:

x "cd %sysfunc(pathname(work))";

The X statement passes commands to an operating system’s command line and they are enclosed in quotes. %sysfunc then is a macro command that allows certain data step functions or call routines as well as some SCL functions to be executed. An example of the latter is pathname and this resolves library or file references and it is interrogating the location of the SAS work library here so it can be passed to the operating systems cd (change directory) command for processing. This method works on Windows and UNIX so Linux should be covered too, offering a certain amount of automation since you don’t have to specify the location of the SAS work library in every session due to the folder name changing all the while.

Of course, if someone were to tell me of another way to declare the location of the generated PNG files that works with RTF and PDF ODS destinations, then I would be all ears. Even direct output without image file creation would be even better. Until then though, the above will do nicely.

Copying only updated new or updated files by command line in Linux or Windows

2nd August 2014

With a growing collection of photographic images, I often find myself making backups of files using copy commands and the data volumes are such that I don’t want to keep copying the same files over and over again so incremental file transfers are what I need. So commands like the following often get issued from a Linux command line:

cp -pruv [source] [destination]

Because this is in Linux, it the bash shell that I use so the switches may not apply to others like ssh, fish or ksh. For my case, p preserves file properties such as its time and date and the cp command does not do this always so it needs adding. The r switch is useful because the copy then in recursive so only a directory needs to specified as the source and the destination needs to be one level up from a folder with the same name there so as to avoid file duplication. It is the u switch that makes the file copy incremental and the v one issues messages to the shell that show how the copying is going. Seeing a file name issued by the latter does tell you how much more needs to be copied and that the files are going where they should.

What inspired this post though is my need to do the same in a Windows session and issuing xcopy commands will achieve the same end. Here are two that will do the needful:

xcopy [source] [destination] /d /s

xcopy [source] [destination] /d /e

In both cases, it is the d switch that ensures that the copy is incremental and you can add a date too, with a colon between it and the /d, if you see fit. The s switch copies only directories that contain files while the e one copies even empty directories. Using the d switch without either of those did not trigger any copying action when I tried so I reckon that you cannot do without either of them. By default, both of these commands issue output to the command line so you can keep an eye on what is happening and this especially is useful when ensuring that files are going to the right destination because the behaviour differs from that of the bash shell in Linux.

Sorting out a system update failure for FreeBSD

3rd April 2014

With my tendency to apply Linux updates using the command, I was happy to see that something similar was possible in FreeBSD too. The first step is to fire up a terminal session and drop into root using the su command. That needs the root superuser password in order to continue and the next step is to update the local repositories using the following command:

pkg update

After that, it is time download updated packages and install these by issuing this command:

pkg upgrade

Most of the time, that is sufficient but I discovered that there are times when the above fails and additional interventions are needed. What I had uncovered were dependency error messages and I set to looking around the web for remedies to this. One forum question that was similar to what I had met with the suggestion of consulting the file called UPDATING in /usr/ports/. An answer like that looks unhelpful but for the inclusion of advice where extra actions were needed. Also, there is a useful article on updating FreeBSD ports that gives more in the way of background knowledge so you understand the more about what needs doing.

Following both that and the UPDATING  file resulted in my taking the following sequence of steps. The first act was to download and initialise the Ports Collection, a set of build instructions.

portsnap fetch extract

The above is a one time only action so future updates are done as follows:

portsnap fetch update

With an up to date Ports Collection in place, it was time to install portman:

pkg install portman

A look through /usr/ports/UPDTAING revealed the commands I needed for updating Python and Perl to address the dependency problem that I was having:

portmaster -o devel/py-setuptools27 devel/py-setuptools
portmaster -r py\*setuptools

With those completed, I re-ran pkg update again and all was well. The extra actions needed to get that result will not get forgotten and I am sharing them on here so I know where they are. If anyone else has use for them, that would be even better.

Installing FreeBSD in a VirtualBox Virtual Machine

2nd March 2014

With UNIX being the basis of Linux, I have a soft spot for trying out any UNIX that can be installed on a PC. For a while, I had OpenSolaris on the go and even vaguely recall having a look at one of the BSD’s. However, any recent attempt to install one of the latter, and there are quite a few around now, got stymied by some sort of kernel panic caused by using AMD CPU’s. With the return to the Intel fold arising from the upgrade of my main home PC last year, it perhaps was time to try again.

The recent release of FreeBSD 10.0 was the cue and I downloaded a DVD image for a test installation in a VirtualBox virtual machine with 4 GB of memory and a 32 GB virtual hard drive attached (expanding storage was chosen so not all the allocated space has been taken so far). The variant of FreeBSD chosen was the 64-bit x86 one and I set to installing it in there. Though not as pretty in appearance as those in various Linux distros, the installer was not that user unfriendly to me. Mind you, I have experience of installing Arch Linux so that might have acclimatised me somewhat.

Those installation screens ask about the keyboard mapping that you want and I successfully chose one of the UK options. There was limited opportunity for adding extras though there was a short list of few from which I made some selections. User account set up also was on offer and I would have been better off knowing what groups to assign for my personal user account so as to have to avoid needing to log in as root so often following system start up later. Otherwise, all the default options were sufficient.

When the installation process was complete, it was time to boot into the new system and all that was on offer was a command line log in session. After logging in as root, it was time to press pkg into service in order to get a desktop environment in place. The first step was to install X:

pkg install xorg

Then, it was time to install a desktop environment. While using XFCE or KDE were alternatives, I chose GNOME 2 due to familiarity and more extensive instructions on the corresponding FreeBSD handbook page. Issuing the following command added GNOME and all its helper applications:

pkg install gnome2

So that GNOME starts up at the next reboot, some extra steps are needed. The first of these is to add the following line into /etc/fstab:

proc /proc procfs rw 0 0

Then, two lines were needed in /etc/rc.conf:

gdm_enable=”YES”
gnome_enable=”YES”

The first enables the GNOME display manager and the second activates other GNOME programs that are needed for a desktop session to start. With each of these in place, I got a graphical login screen at the next boot time.

With FreeDSB being a VirtualBox Guest, it was time to consult the relevant FreeBSD manual page. Here, there are sections for a number of virtual machine tools so a search was needed to find the one for VirtualBox. VirtualBox support for FreeBSD is incomplete in that there is no installation media for BSD systems though Linux and Solaris are supported along with Windows. Therefore, it is over to the FreeBSD repositories for the required software:

pkg install virtualbox-ose-additions

Aside from the virtual machine session not capturing and releasing the mouse pointer automatically, that did everything that was needed even if it was the open source edition of the drivers and their proprietary equivalents. To resolve the mouse pointer issue, I needed to temporarily disable the GNOME desktop session in /etc/rc.conf to drop down to a console only session where xorg. conf could be generated using the following commands:

Xorg -configure
cp xorg.conf.new /etc/xorg.conf

In the new xorg.conf file, the mouse section needs to be as follows:

Section “InputDevice”
Identifier  “Mouse0”
Driver      “vboxmouse”
EndSection

If it doesn’t look like the above and it wasn’t the case for me then it needs changing. Also, any extra lines from the default set up also need removing or the mouse will not function as it should. The ALT+F1 (for accessing GNOME menus) and ALT+F2 (for running commands) keyboard shortcuts then become crucial when your mouse is not working as it should and could avert a panic too; knowing that adjusting a single configuration file will fix a problem when doing so is less accessible is not a good feeling as I discovered to my own cost. The graphics settings were fine by default but here’s what you should have in case it isn’t for you:

Section “Device”
### Available Driver options are:-
### Values: <i>: integer, <f>: float, <bool>: “True”/”False”,
### <string>: “String”, <freq>: “<f> Hz/kHz/MHz”
### [arg]: arg optional
Identifier  “Card0”
Driver      “vboxvideo”
VendorName  “InnoTek Systemberatung GmbH”
BoardName   “VirtualBox Graphics Adapter”
BusID       “PCI:0:2:0”
EndSection

The next step is to ensure that your HAL settings are as they should. I needed to create a file in /usr/local/etc/hal/fdi/policy called 90-vboxguest.fdi that contains the following:

<?xml version=”1.0″ encoding=”utf-8″?>
<!--
# Sun VirtualBox
# Hal driver description for the vboxmouse driver
# $Id: chapter.xml,v 1.33 2012-03-17 04:53:52 eadler Exp $
Copyright (C) 2008-2009 Sun Microsystems, Inc.
This file is part of VirtualBox Open Source Edition (OSE, as
available from http://www.virtualbox.org. This file is free software;
you can redistribute it and/or modify it under the terms of the GNU
General Public License (GPL) as published by the Free Software
Foundation, in version 2 as it comes in the “COPYING” file of the
VirtualBox OSE distribution. VirtualBox OSE is distributed in the
hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa
Clara, CA 95054 USA or visit http://www.sun.com if you need
additional information or have any questions.
-->
<deviceinfo version=”0.2″>
<device>
<match key=”info.subsystem” string=”pci”>
<match key=”info.product” string=”VirtualBox guest Service”>
<append key=”info.capabilities” type=”strlist”>input</append>
<append key=”info.capabilities” type=”strlist”>input.mouse</append>
<merge key=”input.x11_driver” type=”string”>vboxmouse</merge>
<merge key=”input.device” type=”string”>/dev/vboxguest</merge>
</match>
</match>
</device>
</deviceinfo>

With all that set, it is time to ensure that the custom user account is added to the wheel and operator groups using this command:

pw user mod [user name] -G wheel operator

Executing the above as root means that the custom account can run the su command so that logging in as root at the start of a desktop session no longer is needed. That is what being in the wheel group allows and the anyone in the operator group can shut down or restart the system. Both are facilities readily available in Linux so I fancied having them in FreeBSD too.

Being able to switch to root in a terminal session meant that I could go on to add software like Firefox, Libreoffice, GIMP, EMACS, Geany, Netbeans, Banshee and so on. There may be a line of opinion that FreeBSD is a server operating system but all of these make it more than passable for serving as a desktop one too. There may be no package management GUI as such and the ones that come with GNOME do not work either but anyone familiar with command line working will get around that.

FreeBSD may be conservative but that has its place too and being able to build up a system one item at a time teaches far more than getting everything already sorted in one hit. So far, there is enough documentation to get me going and I hope to see where else things go too. So far, the OS hasn’t been that intimidating and that’s good to see.

Setting up a WD My Book Live NAS on Ubuntu GNOME 13.10

1st December 2013

The official line from Western Digital is that they do not support the use of their My Book Live NAS drives with Linux or UNIX. However, what that means is that they only develop tools for accessing their products for Windows and maybe OS X. It still doesn’t mean that you cannot access the drive’s configuration settings by pointing your web browser at http://mybooklive.local/. In fact, not having those extra tools is no drawback at all since the drive can be accessed through your file manager of choice under the Network section and the default name is MyBookLive too so you easily can find the thing once it is connected to a router or switch anyway.

Once you are in the servers web configuration area, you can do things like changing its name, updating its firmware, finding out what network has been assigned to it, creating and deleting file shares, password protecting file shares and other things. These are the kinds of things that come in handy if you are going to have a more permanent connection to the NAS from a PC that runs Linux. The steps that I describe have worked on Ubuntu 12.04 and 13.10 with the GNOME  desktop environment.

What I was surprised to discover that you cannot just set up a symbolic link that points to a file share. Instead, it needs to be mounted and this can be done from the command line using mount or at start-up with /etc/fstab. For this to happen, you need the Common Internet File System utilities and these are added as follows if you need them (check on the Software Software or in Synaptic):

sudo apt-get install cifs-utils

Once these are added, you can add a line like the following to /etc/fstab:

//[NAS IP address]/[file share name] /[file system mount point] cifs
credentials=[full file location]/.creds,
iocharset=utf8,
sec=ntlm,
gid=1000,
uid=1000,
file_mode=0775,
dir_mode=0775
0 0

Though I have broken it over several lines above, this is one unwrapped line in /etc/fstab with all the fields in square brackets populated for your system and with no brackets around these. Though there are other ways to specify the server, using its IP address is what has given me the most success and this is found under Settings > Network on the web console. Next up is the actual file share name on the NAS and I have used a custom them instead of the default of Public. The NAS file share needs to be mounted to an actual directory in your file system like /media/nas or whatever you like and you will need to create this beforehand. After that, you have to specify the file system and it is cifs instead of more conventional alternatives like ext4 or swap. After this and before the final two space delimited zeroes in the line comes the chunk that deals with the security of the mount point.

What I have done in my case is to have a password protected file share and the user ID and password have been placed in a file in my home area with only the owner having read and write permissions for it (600 in chmod-speak). Preceding the filename with a “.” also affords extra invisibility. That file then is populated with the user ID and password like the following. Of course, the bracketed values have to be replaced with what you have in your case.

username=[NAS file share user ID]
password=[NAS file share password]

With the credentials file created, its options have to be set. First, there is the character set of the file (usually UTF-8 and I got error code 79 when I mistyped this) and the security that is to be applied to the credentials (ntlm in this case). To save having no write access to the mounted file share, the uid and gid for your user needs specification with 1000 being the values for the first non-root user created on a Linux system. After that, it does no harm to set the file and directory permissions because they only can be set at mount time; using chmod, chown and chgrp later on has no effect whatsoever. Here, I have set permissions to read, write and execute for the owner and the user group while only allowing read and execute access for everyone else (that’s 775 in the world of chmod).

All of what I have described here worked for me and had to gleaned from disparate sources like Mount Windows Shares Permanently from the Ubuntu Wiki, another blog entry regarding the permissions settings for a CIFS mount point and an Ubuntu forum posting on mounting CIFS with UTF-8 support. Because of the scattering of information, I just felt that it needed to all together in one place for others to use and I hope that fulfils someone else’s needs in a similar way to mine.

Sorting out hogging of the Super (or Windows) Key by GNOME Shell

12th November 2013

Most of the time, GNOME Shell’s use of the Super (or Windows) key on a standard keyboard to open up its dash area is no issue and is a handy counterpart to what you might do in Windows, especially in its latest incarnations. However, it does cause trouble if you are using a VirtualBox virtual machine with Windows installed in there. While VMware Player is immune to this problem I thought that I would see if there was a workaround for it.

The issue might arise from VirtualBox’s non-grabbing of the Super key like others but a solution can be found in GNOME itself. Opening up dconf-editor and navigating to org > gnome > mutter. In there, you will find a setting called overlay-key and this can be changed. One option is to delete the SUPER_L value and leave it that way. My own preference was to set it to a different key and, to do that, I needed to know what the various key identifiers were. To get these, I ran the following command (just replace any quotes with alternatives in the shell before executing this):

xev | grep -A2 --line-buffered ‘^KeyRelease’ | sed -n ‘/keycode /s/^.*keycode \([0-9]*\).* (.*, \(.*\)).*$/\1 \2/p’

This opened up an Event Tester window that will need closing when testing is complete. More importantly, the aliases for any keys that were pressed to are issued to the terminal session so you can see what’s what. Initially, the one for the Alt Gr key appealed to me and I set “ISO_Level3_Shift” as the value of the overlay-key property in dconf-editor. When that didn’t work, I set the value to “Menu” and it behaved as expected. While this will mean that context menus will have to be accessed by right-clicking in a Windows session, that is what I do anyway so there isn’t going to be much of a loss in what I have done. A function key might have been another option but I reckon that the context menu key will do me.

  • All the views that you find expressed on here in postings and articles are mine alone and not those of any organisation with which I have any association, through work or otherwise. As regards editorial policy, whatever appears here is entirely of my own choice and not that of any other person or organisation.

  • Please note that everything you find here is copyrighted material. The content may be available to read without charge and without advertising but it is not to be reproduced without attribution. As it happens, a number of the images are sourced from stock libraries like iStockPhoto so they certainly are not for abstraction.

  • With regards to any comments left on the site, I expect them to be civil in tone of voice and reserve the right to reject any that are either inappropriate or irrelevant. Comment review is subject to automated processing as well as manual inspection but whatever is said is the sole responsibility of the individual contributor.